# Litmir EPUB daily extraction pattern

Use this reference when the user uploads a Litmir-style `.epub` and wants a recurring daily digest/extraction pipeline.

## Recognition

- EPUB is a ZIP archive with files such as:
  - `mimetype`
  - `META-INF/container.xml`
  - `content.opf`
  - `toc.ncx`
  - `content0.html`, `content1.html`, etc.
  - optional `cover.xhtml`, `cover.jpg`, fonts, CSS.
- The book text is usually in the OPF spine order, not necessarily in filename sort order.

## Extraction approach

1. Copy the uploaded EPUB into a stable project dir, e.g. `/home/spazio/<book_project>/source/original.epub`.
2. Open EPUB with Python `zipfile`.
3. Parse `content.opf` with `xml.etree.ElementTree` using the OPF namespace:
   - build manifest by item `id`;
   - read spine `itemref idref` order;
   - select items with media type `application/xhtml+xml` or `text/html`.
4. Extract readable text from each HTML/XHTML file with `html.parser.HTMLParser` or a proper HTML parser if available.
5. Normalize whitespace and HTML entities.
6. Write full text to `text/book_full.txt`.

## Important pitfall: cover-only fragments

Litmir EPUBs often include `cover.xhtml`, which can create a useless first fragment such as `Cover of ...` with fewer than 100 characters.

Before creating final fragments:

- skip `cover.xhtml` / cover-only sections;
- or rebuild fragments after extraction and remove a leading block matching `## Cover of ...`;
- verify the first fragment contains real book text, not metadata/cover only.

## Chunking

For short/medium EPUBs, prefer paragraph chunks of ~7k–8k characters so each daily digest has enough density without overwhelming the model. Avoid chapter-size chunks if the source has only a few huge HTML files.

Suggested project outputs:

- `manifest.json` with `book`, `source`, `full_text`, `total_fragments`, and fragment records;
- `state.json` with `next_fragment: 1`;
- `daily_fragments/fragment_001.txt`, etc.;
- `daily_outputs/` for generated reports;
- `TOOLS_AND_INSIGHTS.md` for cumulative tools/questions/Cosmos meanings.

## Stateful cron script

Create `~/.hermes/scripts/<book>_next_fragment.py` that:

- reads project `manifest.json` and `state.json`;
- supports `--peek` without advancing state;
- emits JSON with `status`, `book`, `fragment_number`, `total_fragments`, `fragment_title`, `fragment_path`, `output_path`, `accumulating_file`, and `fragment_text`;
- advances `next_fragment` only on real cron runs.

## Timezone handling for Margarita

If Margarita asks for a daily digest at a clock time and does not specify timezone, default to her usual local Bali/Samui framing when that context is active. Convert to scheduler UTC and verify `next_run_at`. Example: `16:30` UTC+7 = `30 9 * * *`.

## Confirmation nuance

If a previous turn asked her to resend/provide the file and she repeats the same scheduling request together with the document upload, treat that as sufficient confirmation to proceed with extraction, file writes, and cron creation. Do not ask a second time unless timezone or scope genuinely changes.