PumpkinDrama@reddthat.com

PumpkinDrama@reddthat.com

PyFed (PieFed) Codebase Analysis — Comprehensive Feature Specification

Project: PyFed v1.8.0-dev (PieFed) — Python Flask-based ActivityPub link aggregation platform (Lemmy/Mbin alternative) Codebase Location: /home/user/code/python/pyfedi/ Target: Threadlight — Rust-based single-instance social platform with trust-based governance, credit economy, circles, and community forking

1. Architecture Overview

1.1 Stack

Framework: Flask + SQLAlchemy ORM
Database: PostgreSQL (primary), SQLite (dev)
Cache: Redis, FileSystemCache
Queue: Celery + Redis broker
Frontend: Jinja2 templates + HTMX + Bootstrap
Auth: Flask-Login + JWT + OAuth (Google, Mastodon, Discord)
Federation: ActivityPub (custom implementation)
Search: sqlalchemy-searchable (TSVector full-text search)

1.2 Directory Layout

pyfedi/
├── app/
│   ├── activitypub/    # Federation logic (actors, routes, signature, util)
│   ├── admin/          # Admin panel (routes, forms, util, constants)
│   ├── api/alpha/      # REST API (FastAPI-backed swagger-documented API)
│   ├── auth/           # Login, registration, OAuth, passkeys, onboarding
│   ├── chat/           # Private messaging
│   ├── community/      # Community management (CRUD, moderation, wiki)
│   ├── dev/            # Developer tools
│   ├── domain/         # Domain management
│   ├── feed/           # Curated feeds (user collections of communities)
│   ├── instance/       # Instance exploration
│   ├── main/           # Home page, global routes, RSS
│   ├── post/           # Post/reply CRUD, voting, bookmarks, embeds
│   ├── search/         # Full-text search
│   ├── tag/            # Tag browsing
│   ├── topic/          # Topic-based community grouping
│   ├── user/           # Profile, settings, filters, blocks
│   ├── shared/         # Shared service layer (tasks, upload, feed, auth, etc.)
│   ├── models.py       # All database models (~4273 lines)
│   ├── utils.py        # Utility functions (~4602 lines)
│   ├── constants.py    # App-wide constants
│   ├── inoculation.py  # Media literacy / anti-radicalization resources
│   └── plugins/        # Plugin system
├── migrations/         # Alembic DB migrations
├── tests/              # Test suite
└── docs/               # Documentation

2. Content Filtering System (Multi-Layered)

2.1 Keyword Filters (`Filter` Model)

DB Model (app/models.py:3509-3524):

class Filter(db.Model):
    id            = Integer, PK
    title         = String(50)           # User-given name
    filter_home   = Boolean, default=True # Apply to home feed
    filter_posts  = Boolean, default=True # Apply to community posts
    filter_replies = Boolean, default=False # Apply to replies
    hide_type     = Integer, default=0    # 0 = semi-transparent, 1 = hide completely
    user_id       = Integer, FK->User
    expire_after  = Date                  # Optional expiration
    keywords      = String(500)           # Newline-separated keywords

Implementation (app/utils.py:1997-2036):

Three cached functions: user_filters_home(), user_filters_posts(), user_filters_replies()
Each loads filters where filter_{scope}=True and expire_after > today OR expire_after IS NULL
Keywords split by newline, lowercased
hide_type == 0 → posts get semi-transparent treatment in templates
hide_type == 1 → posts excluded from SQL query (template-level filtering)
Result structure: {filter_title: {set_of_keywords}, '-1': {set_of_keywords}} where -1 means “hide completely”

2.2 Content Visibility Settings (User Model)

On User model (app/models.py:959-1078):

Field	Type	Default	Description
`hide_nsfw`	Integer	1	0=show, 1=hide, 2=blur, 3=semi-transparent
`hide_nsfl`	Integer	1	Same scale
`hide_gen_ai`	Integer	2	0=show, 1=hide, 2=label, 3=semi-transparent
`ignore_bots`	Integer	0	0=show, 1=hide completely, 2=blur, 3=semi-transparent
`hide_low_quality`	Boolean	false	Hide posts from low_quality communities
`hide_read_posts`	Boolean	false	Hide posts user has already interacted with
`read_language_ids`	ARRAY(Integer)	[]	Only show content in these languages
`community_keyword_filter`	String(150)	‘’	Hide communities with these words in name

Template-level filtering (app/templates/user/edit_filters.html):

hide_type_choices: Show, Hide completely, Blur thumbnail, Make semi-transparent
ai_hide_type_choices: Show, Hide completely, Label as AI, Make semi-transparent

2.3 Blocking System (4 Levels)

Block Target	DB Model	Primary Key	Implementation
User	`UserBlock`	`blocker_id, blocked_id`	SQL exclusion in feed queries
Community	`CommunityBlock`	`user_id, community_id`	SQL exclusion in feed queries
Domain	`DomainBlock`	`user_id, domain_id`	`Post.domain_id NOT IN blocked_domains()`
Instance	`InstanceBlock`	`user_id, instance_id`	`Post.instance_id NOT IN blocked_instances()`

Implementation (app/utils.py:3236-3301):

All blocking is applied at the SQL query level in get_deduped_post_ids()
Blocked users: p.user_id NOT IN :blocked_accounts
Blocked communities: p.community_id NOT IN :blocked_community_ids
Blocked domains: (p.domain_id NOT IN :domain_ids OR p.domain_id IS NULL)
Blocked instances: (p.instance_id NOT IN :instance_ids OR p.instance_id IS NULL)
Cached via @cache.memoize(timeout=300) helper functions

2.4 Flair Blocking (`CommunityFlairBlock`)

DB Model (app/models.py:4115-4119):

class CommunityFlairBlock(db.Model):
    community_id      = Integer, FK->Community
    community_flair_id = Integer, FK->CommunityFlair
    user_id           = Integer, FK->User

Users can block posts with specific flairs/tags within a community. Applied in get_deduped_post_ids():

p.id NOT IN (SELECT post_id FROM post_flair WHERE flair_id IN :blocked_flair_ids)

3. Vote / Score System

3.1 Vote Models

PostVote (app/models.py:3459-3475):

class PostVote(db.Model):
    id         = Integer, PK
    user_id    = Integer, FK->User     # Who voted
    author_id  = Integer, FK->User     # Post author (reputation target)
    post_id    = Integer, FK->Post
    effect     = Float                 # Vote weight (not just -1/+1)
    emoji      = String(50)            # Emoji reaction
    created_at = DateTime

PostReplyVote (app/models.py:3478-3494): Same structure for comment votes.

Vote effects are floats, enabling weighted voting systems (reputation-weighted, etc., though PyFed uses simple +/-1 in practice with float storage for future).

3.2 Vote Federating / Privacy

Setting	Location	Effect
`federate_votes`	User setting	When false, votes only count locally, not broadcast via ActivityPub
`vote_privately`	User model	When true, votes are hidden from other users
`show_scores`	API schema	Controls display of aggregate scores
Instance `votes_are_public()`	Instance model	Returns True for Lemmy/Mbin/kbin (for compatibility)

3.3 Downvote Controls (Granular)

Site-level (Site.enable_downvotes, Boolean, default=True):

Master toggle: if false, no downvoting anywhere on instance

Community-level (Community.downvote_accept_mode, Integer):

DOWNVOTE_ACCEPT_NONE    = -1  # No downvotes allowed
DOWNVOTE_ACCEPT_ALL     = 0   # Anyone can downvote
DOWNVOTE_ACCEPT_MEMBERS = 2   # Only community members
DOWNVOTE_ACCEPT_INSTANCE= 4   # Only users from same instance
DOWNVOTE_ACCEPT_TRUSTED = 6   # Only from trusted instances

User-level restrictions (app/utils.py:1766-1807):

Banned users can’t downvote
Bot accounts can’t downvote
Users with negative attitude (more downvotes than upvotes cast) can’t downvote
Users with reputation < -10 can’t downvote
Banned from community can’t downvote there

3.4 Vote Quota

VOTE_QUOTA config: max 240 votes per day (configurable)
votes_cast_today(user_id): Redis counter, resets daily
Displayed on user profiles as vote_quota_used

3.5 Reputation & Attitude

User fields:

reputation (Float): Accumulated reputation score
attitude (Float, -1 to 1): Ratio of (upvotes_cast - downvotes_cast) / total_votes_cast
- Only calculated after 10+ votes
- Used to determine if user can downvote

Reputation calculation:

Based on recent 50 post votes + 50 comment votes
Post authors’ reputation affected by votes on their content
Low-quality communities marked with low_quality flag don’t improve reputation

3.6 Reply Visibility Thresholds

Setting	Description
`reply_collapse_threshold` (integer, default=-10)	Replies scored at or below this are collapsed
`reply_hide_threshold` (integer, default=-20)	Replies scored at or below this are hidden entirely

Comment sort options: hot, top, new, old

4. Hide Read / Hidden Posts

4.1 Read Posts Tracking

Many-to-many table (read_posts, app/models.py:928-933):

read_posts = db.Table('read_posts',
    db.Column('user_id', Integer, FK->User, PK),
    db.Column('read_post_id', Integer, FK->Post, PK),
    db.Column('interacted_at', DateTime, indexed),
    Index('ix_read_posts_user_post', 'user_id', 'read_post_id')
)

User model relationship:

User.read_post: posts the user has read/interacted with
Post.read_by: users who have read this post
User.hide_read_posts (Boolean): master toggle

Hide Read in Feed Query (app/utils.py:3264-3265):

p.id NOT IN (SELECT read_post_id FROM "read_posts" WHERE user_id = :user_id)

4.2 Hidden Posts

Table (hidden_posts, app/models.py:937-942): Same structure as read_posts.

Always applied (app/utils.py:3268):

p.id NOT IN (SELECT hidden_post_id FROM "hidden_posts" WHERE user_id = :user_id)

Users can manually hide individual posts. These are always excluded from feed.

4.3 Interaction Recording

Interaction is recorded when:

A user opens/clicks a post
A user upvotes or downvotes a post
This powers both hide_read_posts and the “read history” feature

4.4 View Templates

user/read_posts.html: shows user’s read post history
user/hidden_posts.html: shows hidden posts with unhide option
user/_read_posts_nav.html: navigation for read/hidden posts

5. Content Moderation & Governance

5.1 Community Moderation

Community membership levels:

SUBSCRIPTION_OWNER     = 3
SUBSCRIPTION_MODERATOR = 2
SUBSCRIPTION_MEMBER    = 1
SUBSCRIPTION_NONMEMBER = 0
SUBSCRIPTION_PENDING   = -1
SUBSCRIPTION_BANNED    = -2

Moderation tools:

Community bans (with reason, optional duration)
Post/reply removal
Report handling (new → escalated → resolved)
Moderation log (audit trail)
Content retention policies (auto-delete after N months)
Flair management (text/color/blur-images)
Wiki management with revision history
Mod-only communities

5.2 Report System

Report types (constants.py):

REPORT_TYPE_USER      = 0
REPORT_TYPE_POST      = 1
REPORT_TYPE_REPLY     = 2
REPORT_TYPE_COMMUNITY = 3
REPORT_TYPE_MESSAGE   = 4

Report states: new → escalated → appealed → resolved → discarded

Report reasons (from user/forms.py):

Breaks community rules
Harassment
Threatening violence
Promoting hate/genocide
Minor abuse/sexualization
Sharing personal info (doxing)
Spam
Non-consensual intimate media
Prohibited transaction
Impersonation
Copyright violation
Trademark violation
Self-harm/suicide
Other
Misinformation/disinformation
Racism/sexism/transphobia
Malicious reporting

5.3 Instance-level Controls

Setting	Type	Description
`enable_downvotes`	Boolean	Master downvote toggle
`registration_mode`	String	Open / RequireApplication / Closed
`application_question`	Text	Question for registration applications
`private_instance`	Boolean	Require login to view anything
`allow_or_block_list`	Integer	1=allowlist, 2=blocklist
`allowlist_mode`	Integer	0=weak, 1=strong, 2=intense
`blocked_phrases`	Text	Discard incoming content with these phrases
`auto_decline_referrers`	Text	Auto-reject registrations from these referrers
`honeypot`	Boolean	Bot detection

5.4 Inoculation System

app/inoculation.py contains educational content displayed to users about:

Groupthink and cult dynamics
Alt-right radicalization pipelines
Media literacy
Confirmation bias
The dark side of upvotes

This is shown at registration and optionally in sidebars — a unique approach to platform responsibility.

6. Custom Feeds & Curated Collections

6.1 Feed System

Feeds are user-curated collections of communities, acting like topic-based aggregations.

Feed model (app/models.py):

User-owned (or instance-wide)
Can be public or private
Auto-follow/auto-leave behavior for subscribing
Remote feeds via ActivityPub federation
Feeds can have parent-child hierarchy (sub-feeds)

6.2 Topic System

Topics are admin-created groupings of communities with hierarchical structure (Topic model has parent_id).

6.3 Community Features

Private communities: Only members can view
Local-only communities: No federation
Encrypted communities: End-to-end encryption
Restricted-to-mods posting: Only moderators can post
Question/Answer mode: StackOverflow-style communities (question_answer flag)
Default layout: Per-community layout preference
Community themes: Visual customization per community
Post flairs: Categorization with visual styling (text color, background color, blur images)

7. User Experience & Accessibility

7.1 Themes & Visual

18+ themes including accessibility-focused ones (high contrast, dyslexic font)
Font choices: Atkinson Hyperlegible, OpenDyslexic, Inter, Roboto
Compact UI modes (3 levels: normal/compact/ultra-compact)
Low bandwidth mode (reduces page size)
Code syntax highlighting (18 color schemes)
Additional CSS injection for users and admins

7.2 Productivity Features

Usage limits: max_hours_per_day with configurable change restrictions
Vote quota: Limits daily voting to prevent vote manipulation
Scheduled posts: Posts with scheduled_for, repeat (daily/weekly/monthly)
Post drafts: Posts with status -1 (draft)
Bookmarks: Separate for posts and replies
Cross-posting: Track cross-posted content across communities
User notes: Private notes about other users
Keyboard shortcuts: Documented in keyboard_shortcuts.html

7.3 Notification System

Granular notification subscriptions:

User follows (when followed user posts)
Community posts (new posts in community)
Topic posts (new posts in topic communities)
Post replies (new top-level comments)
Reply replies (replies to your comments)
Mentions
Private messages
Ban/unban notifications
New moderator notifications
Report updates

8. Key Technical Patterns

8.1 Feed Query Pipeline

The get_deduped_post_ids() function (app/utils.py:3197-3317) is the core query builder:

View filter selection (subscribed/local/popular/all/moderating)
Community source selection
Following user content inclusion (optional)
Content type filters (nsfw, nsfl, ai, bots)
Hide read posts
Hidden posts exclusion
Language filter
Blocked domains/instances/communities/users
Community bans
Flair blocks
Community keyword filter
Sort application (hot/scaled/top/new/active/old)

8.2 Vote Quota Circuit (Redis-based)

def votes_cast_today(user_id):
    num = redis_client.get(f'votes_cast_{date.today()}_{user_id}')
    if num is None: return 0
    return int(num)

8.3 Deduplication Strategy

Cross-posts are tracked via Post.cross_posts (ARRAY of post IDs) and deduplicated in the feed query:

Posts with the same URL are grouped
Non-bot posts preferred over bot posts
Post with most replies preferred
Seen posts tracked in seen_before set

9. Features Valuable for Threadlight → Detailed Recommendations

9.1 HIGH VALUE — Direct Port

9.1.1 Multi-Layered Content Filtering System

Why: Threadlight’s goal of encouraging quality engagement requires giving users granular control over their feed.