Piefed Subscription Feed Tracker & Outlier Poster

submitted by

I wanted to do something like this, do you know if there is something like pythorhead I can use for piefed or where can I find the API docs?

#### **1. Subscription Feed Tracker Script**
- **Objective**: Track all posts from subscribed communities on Piefed.
- **Features**:
  - Save metadata for each post (title, author, community, URL, timestamp, score, etc.).
  - Calculate the moving average and standard deviation of post scores per community.
  - Identify outliers (posts with scores significantly above the moving average, e.g., >2 standard deviations).
  - Allow user to specify the number of top outliers to retrieve.

#### **2. Outlier Poster Script**
- **Objective**: Automatically post identified outliers to a user-specified Piefed community.
- **Features**:
  - Avoid reposting duplicates (check URL history, even for crossposts).
  - Log posted URLs to prevent reposting.
  - Support customizable thresholds for outlier detection.

#### **Tech Stack Suggestions**:
- **Language**: Python (for data processing and API interaction).
- **Libraries**:
  - `requests` or `httpx` for API calls.
  - `pandas` for data analysis (moving average, standard deviation).
  - `sqlite3` or `pymongo` for storing post metadata and history.

#### **Next Steps**:
- Research Piefed’s API documentation for endpoints and rate limits.
- Draft a schema for storing post data and history.
- Implement the outlier detection logic.
- Build the poster script with duplicate-checking.
1

Log in to comment

Hot Top New Old

As mentioned already, our API links are in the codeberg: stable and dev branches.

There isn't yet a pythorhead-like library that I know of, so API requests would need to be done using an http library. As much as I love requests, I believe it is largely unmaintained at this point, so something like httpx might be better (though that is having issues now too, we'll see). The general workflow for interacting with the API is to make a network request, getting a json back. Then, parse the json in python exactly like you would a dictionary (or nested dictionaries).

Also, you might want to check out the work done over at !lemmydirectory@lemmy.dbzer0.com to see how they do things.

 reply
1
edited

I grabbed the API for the stable branch from the codeburg repository. The unstable branch is there too if you want that instead.

Edit: The non-outlier data would also be interesting to look at, if you'd feel like making a post about it when you're done.

 reply
1