rtiktoken

A thin wrapper around the tiktoken-rs crate, allowing to encode text into Byte-Pair-Encoding (BPE) tokens and decode tokens back to text. This is useful to understand how Large Language Models (LLMs) perceive text.

Last 30 days

This package has been downloaded 502 times in the last 30 days. More downloads than an obscure whitepaper, but not enough to bring down any servers. A solid effort! The following heatmap shows the distribution of downloads per day. Yesterday, it was downloaded 43 times.

The following line graph shows the downloads per day. You can hover over the graph to see the exact number of downloads per day.

Last 365 days

This package has been downloaded 2,044 times in the last 365 days. Now we’re talking! This work is officially 'heard of in academic circles', just like those wild research papers on synthetic bananas. The day with the most downloads was Feb 08, 2025 with 50 downloads.

The following line graph shows the downloads per day. You can hover over the graph to see the exact number of downloads per day.