RSS
Sort by: Publisher Date Title
  • Citation Logo
    Is OpenAI imposing a Token Tax on Non-English Languages?
    Publisher: MOR10
    2024-01-31

    Is OpenAI imposing a Token Tax on Non-English Languages?

    The token count on international languages are significantly higher than English.  The post discusses the issue of OpenAI's tokenizer assigning more tokens to non-English languages compared to English, effectively imposing a "token tax" on internat...
    The token count on international languages are significantly higher than English.  The post discusses the issue of OpenAI's tokenizer assigning more tokens to non-English languages compared to English, effectively imposing a "token tax" on internat...

    Tags: openai, localization, costs

    5 1
Page 1 of 1
Powered by Optimal Access

Author Tags