John Costa ef69ab8652 Split long sentences on commas to prevent skipped words (#143) ...

## Summary

When a single sentence exceeds `max_tokens` (50) and has no
sentence-ending punctuation (`.`, `!`, `?`), the text splitting logic
now falls back to splitting on commas, semicolons, and colons. This
prevents the model from silently skipping parts of long sentences.

Fixes #38

2026-03-24 11:50:44 +01:00

..

__init__.py

Copy the files in the new repo for public release

2026-01-12 11:25:16 +01:00

base.py

Copy the files in the new repo for public release

2026-01-12 11:25:16 +01:00

text.py

Split long sentences on commas to prevent skipped words (#143)

2026-03-24 11:50:44 +01:00