Next: , Previous: , Up: feeder  


Storage

Each feed is kept in separate subdirectory in the feeds/. Its name is not important, but human readable is a preferable choice. It contains:

url

File with the URL of the feed. This is the only file you have to manually deal with.

max

If may contain maximal number of messages per current feed to keep and process.

etag, hdr, out

Those files are used by curl to keep the content, its proper mtime (for If-Modified-Since header generation), ETag and response headers for debugging.

feed.zst

It contains the content itself. Compressed with Zstandard.

download.hash, parse.hash

SHA-512 hash of the feed.zst, used to determine if feed was updated and parser has to do the job.

title

Automatically generated file with the title of the feed.

cur/, new/, tmp/

Directories necessary for making the directory Maildir mailbox. They contain parsed news posts.

encs/, warcs/

Optionally created directories for storing of downloaded enclosures and WARCs.