What is a token in claude?

In short tokens are the smallest unit of a language model and corresponds roughly to words but not precisely.

Some examples

· “hello” à usually 1 token
“Fort Collins, Colorado” à ≈ 4–5 tokens
<100 English words> à ≈ 130 tokens

So its close, but not one per one on a word.
Let me run a few tests.

Testing token counting

I found this from claude https://platform.claude.com/docs/en/build-with-claude/token-counting [1] https://platform.claude.com/docs/en/api/messages/count_tokens [2]

In this doc they give an example curl command to get how many tokens an input would be. (The tool is free to use but does have some rate limits) (Of course swap it out with your Key)

If I run this

> curl https://api.anthropic.com/v1/messages/count_tokens \
--header "x-api-key: sk-ant-REPLACE_WITH_YOUR_KEY" \
--header "anthropic-version: 2023-06-01" \
--header "content-type: application/json" \
--data '{
"model": "claude-sonnet-4-6",
"messages": [{"role": "user", "content": "Hello, Claude"}]
}'

I get back

{"input_tokens":10}

10, that seems a bit high it should be 3 or 4?

Let me try with just one work Hi

8 tokens

Let me convert this into a script so I can do some more quick testing.

Token Count Script

> sudo vi /usr/bin/token-count

And place the following in it.

#!/usr/bin/env python3
"""
Simple script to count input tokens using Anthropic's count_tokens endpoint
Works both with piped input and command-line arguments
"""

import sys
import json
import os
import argparse
import requests

def get_content() -> str:
"""Get input either from stdin pipe or command line arguments"""
# Check if anything is being piped in
if not sys.stdin.isatty():
return sys.stdin.read().rstrip()
# Otherwise use command line arguments
parser = argparse.ArgumentParser(description="Count tokens using Anthropic API")
parser.add_argument("text", nargs="*", help="Text to count tokens for")
args = parser.parse_args()

if args.text:
return " ".join(args.text)
print("Error: No input provided", file=sys.stderr)
print("Usage examples:", file=sys.stderr)
print(" echo 'Hello world' | python count_tokens.py", file=sys.stderr)
print(" python count_tokens.py 'Hello world how are you'", file=sys.stderr)
sys.exit(1)

def main():
api_key = os.environ.get("ANTHROPIC_API_KEY") or os.environ.get("ANTHROPIC_API_KEY_FIX")

if not api_key:
print("Error: ANTHROPIC_API_KEY environment variable not set", file=sys.stderr)
sys.exit(1)
content = get_content()

payload = {
"model": "claude-sonnet-4-6", # ΓåÉ change model name if needed
"messages": [
{"role": "user", "content": content}
]
}

headers = {
"x-api-key": api_key,
"anthropic-version": "2023-06-01",
"content-type": "application/json"
}

try:
response = requests.post(
"https://api.anthropic.com/v1/messages/count_tokens",
headers=headers,
json=payload,
timeout=10
)

response.raise_for_status()
data = response.json()

tokens = data.get("input_tokens")

if tokens is not None:
print(tokens)
else:
print("error", file=sys.stderr)
print("Response:", json.dumps(data, indent=2), file=sys.stderr)
sys.exit(1)

if __name__ == "__main__":
main()

You can download it here https://gist.github.com/patmandenver/adc2378f7b1dee821c8e94e410a5e631

Chmod it

> chmod 755 /usr/bin/token-count

Now try it

> token-count Hi
> token-count Hi hi
> token-count 1
> token-count 1 2 3 4

Ok why does hi have 8 tokens rather than 1?

Let me try some more fun

> echo "word_"{001..010} | token-count
> echo "word_"{001..020} | token-count
> echo "word_"{001..100} | token-count

Passing in 10 words then 20 then 100

Averaging 4 tokens a word

What about just passing hi 10, 20, then 100 times

Interesting the overhead seems to be 7+ a token per word.

I guess it is thinking about the numbers in the prior one word_001 seems like 4 things?

OK let me throw some code at it.

Code Testing

I found this one https://github.com/TheAlgorithms/Python [3] that has a collection of simple Python programs that are good examples.

Let’s pull a simple test repo down from github to test it on.

> git clone git@github.com:TheAlgorithms/Python.git
> cd Python/strings

Let’s first look at the lower.py code.
Let me use the wc tool to count the number of words

> wc -w lower.py

That gave me back 93…

Now let me feed it into claude (no questions just feeding it in)

> cat lower.py | token-count

So double the word count.

Let me try another one

> wc -w min_cost_string_conversion.py
> cat min_cost_string_conversion.py | token-count

Roughly 3x in token size vs wc

Testing some real input/outputs

Now let me test some real, costs money, tests.
let me hit the message endpoint https://platform.claude.com/docs/en/api/messages/create [4]

> sudo vi /usr/bin/claude-message

And place the following in it

#!/usr/bin/env python3

"""
Simple script to generate a response using Anthropic's messages endpoint
and report input/output tokens.
Works both with piped input and command-line arguments.
"""

import sys
import json
import os
import argparse
import requests

def get_content() -> str:
"""Get input either from stdin pipe or command line arguments"""
if not sys.stdin.isatty():
return sys.stdin.read().rstrip()

parser = argparse.ArgumentParser(description="Generate response using Anthropic API")
parser.add_argument("text", nargs="*", help="Text prompt for the AI")
args = parser.parse_args()

if args.text:
return " ".join(args.text)

print("Error: No input provided", file=sys.stderr)
print("Usage examples:", file=sys.stderr)
print(" echo 'Hello world' | python generate_response.py", file=sys.stderr)
print(" python generate_response.py 'Tell me a joke'", file=sys.stderr)
sys.exit(1)

def main():

api_key = os.environ.get("ANTHROPIC_API_KEY") or os.environ.get("ANTHROPIC_API_KEY_FIX")

if not api_key:
print("Error: ANTHROPIC_API_KEY environment variable not set", file=sys.stderr)
sys.exit(1)

content = get_content()

payload = {
"model": "claude-sonnet-4-6", # Updated to a real, active model (was "claude-sonnet-4-6")
"messages": [
{"role": "user", "content": content}
],
"max_tokens": 1024, # Adjust as needed
"temperature": 0.7 # Optional: adjust for creativity
}

headers = {
"x-api-key": api_key,
"anthropic-version": "2023-06-01",
"content-type": "application/json"
}

try:
response = requests.post(
"https://api.anthropic.com/v1/messages", # Active endpoint for generating responses
headers=headers,
json=payload,
timeout=30
)

response.raise_for_status()
data = response.json()

# Extract the response content
if "content" in data and data["content"]:
ai_response = data["content"][0]["text"]
print("AI Response:")
print(ai_response)
print("\n---")
else:
raise ValueError("No content in response")

# Extract token usage
usage = data.get("usage", {})
input_tokens = usage.get("input_tokens", "N/A")
output_tokens = usage.get("output_tokens", "N/A")

print(f"Tokens In: {input_tokens}")
print(f"Tokens Out: {output_tokens}")

except requests.RequestException as e:
print("error", file=sys.stderr)
print(f"Request failed: {e}", file=sys.stderr)
if hasattr(e, 'response') and e.response is not None:
print("Response content:", e.response.text, file=sys.stderr)
sys.exit(1)
except (KeyError, ValueError) as e:
print("error", file=sys.stderr)
print(f"Unexpected response format: {e}", file=sys.stderr)
print("Full response:", json.dumps(data, indent=2), file=sys.stderr)
sys.exit(1)

if __name__ == "__main__":
main()

You can download it here https://gist.github.com/patmandenver/2674e45fc50ce8f8feca09325775c2b1

Chmod it

> chmod 755 /usr/bin/claude-message

Now test it

> claude-message "hi"

8 in and 24 out.
And when I check my actual usage I see it increment these exact numbers

Conclusions

Token count is complicated. And I imagine it will change over time.
If you run the scripts I have in here today with the same data I think you will get the same results. However, I do not think that is guaranteed 6 months from now… I imagine in 6 months the Token calculation will change and if you run these same scripts then you may get different token counts.

References

[1]       Token Counting
https://platform.claude.com/docs/en/build-with-claude/token-counting
Accessed 03/2026
[2]       Count tokens in a Message
  https://platform.claude.com/docs/en/api/messages/count_tokens
  Accessed 03/2026
[3]       TheAlgorithms / Python
  https://github.com/TheAlgorithms/Python
  Accessed 03/2026
[4]       Create a Messsage
https://platform.claude.com/docs/en/api/messages/create
  Accessed 03/2026

What is a Token in Claude?

Posted on Friday, March 6, 2026