wordpiece

R Implementation of Wordpiece Tokenization

CRAN Package

Apply 'Wordpiece' (doi:10.48550/arXiv.1609.08144) tokenization to input text, given an appropriate vocabulary. The 'BERT' (doi:10.48550/arXiv.1810.04805) tokenization conventions are used by default.


Documentation


Team


Insights

Last 30 days

Last 365 days

The following line graph shows the downloads per day. You can hover over the graph to see the exact number of downloads per day.

Data provided by CRAN


Binaries


Dependencies

  • Imports7 packages
  • Suggests4 packages
  • Reverse Suggests1 package