Python Tokens
M. Lathika
23H51A66A8
What are Tokens?
• • A token is like a word in Python’s language.
• • Python breaks code into tokens to
understand it.
• • Main types:
• 1. Identifiers
• 2. Literals
• 3. Operators
• 4. Delimiters
Identifiers
• • Names we give to things in code.
• • Used for variables, functions, classes.
• • Rules:
• - Can use letters, numbers, _
• - Cannot start with a number
• - Case-sensitive (Age ≠ age)
• • Examples: name, student_1
Literals
• • Fixed values used in a program.
• • Types:
• - Numbers → 10, 3.5
• - Text (string) → "Hello"
• - Boolean → True, False
• - Special → None
• • Example:
• x = 50
• y = "Hi"
Operators
• • Symbols that do actions.
• • Types:
• - Arithmetic → +, -, *, /
• - Comparison → ==, <, >
• - Logical → and, or, not
• • Example:
• a = 5 + 3 # + adds numbers
Delimiters & Recap
• • Delimiters = symbols that separate code parts
• - (), [], {}, ,, :, ;, " ", #
• • Example:
• list1 = [1, 2, 3]
• • Recap:
• - Identifiers → names
• - Literals → fixed values
• - Operators → do actions
• - Delimiters → separators
Conclusion
• In Python, tokens are the smallest parts of a
program that make it work.
• They include identifiers, literals, operators, and
delimiters.
• Identifiers are the names we give, literals are
fixed values, operators are symbols that
perform actions, and delimiters are symbols
that separate code parts. Understanding
tokens is important because they are the
foundation of writing any Python program.
Thank You
