AI DEVELOPMENT WORKSPACE
Text Chunker
Split exact text into bounded character or word chunks with overlap.
Input is processed on this site only when you run the tool. The application does not save inputs or results.
input.txt
Result result.json
A little clarity goes here.
Run the tool to see your result.
Result preview
UTF-8 text, at most 32768 bytes per input. Tab leaves the editor. Plain text disables syntax colors. Browser textareas may normalize line endings to LF.
{"chunk":{"inputMode":"text","options":{"chunkUnit":{"type":"enum","values":["characters","words"],"default":"characters"},"size":{"type":"integer","min":1,"max":4096,"default":1000},"overlap":{"type":"integer","min":0,"max":4095,"default":100}},"inputLabels":[]}}{"chunk":{"input":"Build useful tools.\nKeep the boundaries clear.\nTest with fictional data.","options":{"chunkUnit":"words","size":6,"overlap":2}}}A QUICK WALKTHROUGH
A familiar workflow,
with room to focus.
Paste text and choose character or word units.
Set a chunk size and a smaller overlap.
Chunk and inspect exact text with byte/code-point/unit ranges.
When to use Text Chunker
Split a text fixture into overlapping character or whitespace-word windows when preparing a downstream processing pipeline.
Read the result
Use exact half-open byte/code-point/unit offsets to reproduce a slice. Character units are code points, not graphemes or tokens; overlap is a mechanical text setting.
SEE IT IN PRACTICE
Worked examples
Sample input. A result you can inspect.
Chunk example
Input
Build useful tools. Keep the boundaries clear. Test with fictional data.
- Chunk units
- Whitespace-delimited words
- Chunk size
- 6
- Overlap units
- 2
Expected result
{"unit":"words","size":6,"overlap":2,"totalUnits":11,"chunks":[{"index":1,"unitStart":0,"unitEnd":6,"byteStart":0,"byteEnd":40,"codePointStart":0,"codePointEnd":40,"overlapUnits":0,"text":"Build useful tools.\nKeep the boundaries "},{"index":2,"unitStart":4,"unitEnd":10,"byteStart":25,"byteEnd":67,"codePointStart":25,"codePointEnd":67,"overlapUnits":2,"text":"the boundaries clear.\nTest with fictional "},{"index":3,"unitStart":8,"unitEnd":11,"byteStart":52,"byteEnd":72,"codePointStart":52,"codePointEnd":72,"overlapUnits":2,"text":"with fictional data."}]}{"kind":"chunks","title":"3 chunks prepared","unit":"words","size":6,"overlap":2,"total":3,"chunks":[{"index":1,"unitStart":0,"unitEnd":6,"byteStart":0,"byteEnd":40,"codePointStart":0,"codePointEnd":40,"overlapUnits":0,"text":"Build useful tools.\nKeep the boundaries "},{"index":2,"unitStart":4,"unitEnd":10,"byteStart":25,"byteEnd":67,"codePointStart":25,"codePointEnd":67,"overlapUnits":2,"text":"the boundaries clear.\nTest with fictional "},{"index":3,"unitStart":8,"unitEnd":11,"byteStart":52,"byteEnd":72,"codePointStart":52,"codePointEnd":72,"overlapUnits":2,"text":"with fictional data."}],"note":"Ranges start at zero and exclude the end position. Full exact chunks remain available in source data.","schema":1}Formats, limits and common errors
Reduce overlap or increase size if the chunk count or expanded output exceeds the cap.
- At most 32,768 UTF-8 bytes, 200 chunks and 262,144 aggregate chunk-text bytes. Excess is rejected, not truncated.
- Size 1–4,096, overlap 0–4,095 and overlap strictly smaller than size. Character means Unicode code point, not a displayed grapheme or token.
- Words are Unicode nonwhitespace runs. Leading whitespace belongs to the first unit and following whitespace to the preceding unit. Whitespace-only text is one unit; empty text gives no chunks. Text is retained exactly.
- Offsets are zero-based, half-open byte, code-point and selected-unit ranges in the submitted text. Browser textareas may normalize line endings. No sentence, semantic, token-budget or retrieval-quality guarantee.
- Input: 32,768 UTF-8 bytes per text field. Output: at most 524,288 UTF-8 bytes; serialized responses have a separate 1 MiB ceiling. Size/work failures return no partial result.