Python

I have encountered a situation where I need to retrieve all posts for a user or community on Lemmy and store them in a database. Due to a recent mishap, I had to revert to an older version of the database, resulting in a loss of some posts.

To overcome this issue, I am seeking guidance on how to efficiently retrieve all posts for a user or community on Lemmy and extract important information such as the URL, Title, Body, and Posted Timestamp.

To accomplish this task, I am utilizing the Pythörhead library, which provides a Python interface for interacting with the Lemmy API.

One approach I have considered is searching for each URL before posting to avoid duplicating posts. However, this method would be very slow and would create a significant load on the instance, potentially impacting performance.

Any suggestions on how best to solve this issue would be invaluable.

For reference, this is the Lemmy API endpoint that can be utilized for retrieving posts is.

Thank you in advance for your assistance!