Slides On Algebra of Coding Theory

Algebra for Coding Theory
B. Sainath
sainath.bitragunta@pilani.bits-pilani.ac.in
Department of Electrical and Electronics Engineering

Birla Institute of Technology and Science, Pilani
September 7, 2019
B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 1 / 30

Outline
1 The Binary Field
2 Groups
3 Rings
4 Fields
5 Vector Spaces
6 Linear Algebra
7 References & Further Reading

The Binary Field
Algebraic theory
provides tools necessary to design encoders & decoders
classifies many arithmetic systems
Binary field
Finite field (Galois Field, GF(2))
Binary field with only two elements
Let symbols 0 and 1 denote the two elements
Operations of addition (XOR) & multiplication (AND) (details in class)
Example (in class)
Hexadecimal field is another popular finite field
contains 16 elements (0, 1, . . . , 9, A, B, C, D, E, F )
Exercise: Prepare addition & multiplication table
GF (q):

The Binary Field
Algebraic theory
provides tools necessary to design encoders & decoders
classifies many arithmetic systems
Binary field
Finite field (Galois Field, GF(2))
Binary field with only two elements
Let symbols 0 and 1 denote the two elements
Operations of addition (XOR) & multiplication (AND) (details in class)
Example (in class)
Hexadecimal field is another popular finite field
contains 16 elements (0, 1, . . . , 9, A, B, C, D, E, F )
Exercise: Prepare addition & multiplication table
GF (q): Finite field with q elements
Finite fields applications: linear block codes (e.g., BCH codes &
Reed–Solomon codes), cryptography algorithms

Imp. Note
GF(q) construction
Finite fields with q elements can be constructed when, and only when,
q = pm
p is a prime number
m is an arbitrary positive integer
GF(2) is the smallest field

Groups & Abelian Group
Definition
Let ∗ denote an operation on pairs of elements
A group (G, ∗) is a set together with an operation ∗ on pairs of elements
in the set satisfying four properties
Closure: For every a, b in the set G, c = a ∗ b is in the set

Definition
Associativity: For every a, b, c in the set a ∗ (b ∗ c) = (a ∗ b) ∗ c

Definition
Identity: There is unique element e called the identity element such that
a∗e =e∗a=a

Definition
a∗e =e∗a=a
Inverse: If a is in the set, then there is some element b in the set called
inverse of a such that a ∗ b = b ∗ a = e

Definition
a∗e =e∗a=a
Order of G: number of elements in G

Finite group: G has finite number of elements

Definition
a∗e =e∗a=a

Commutative property: For all a, b in the group G, a ∗ b = b ∗ a

Definition
a∗e =e∗a=a

Commutative property: For all a, b in the group G, a ∗ b = b ∗ a
Commutative group (abelian group): Groups with commutative property

Examples of Groups
Simple Examples
Set of integers under addition
Set of positive rationals under multiplication
Set of 2 × 2 real-valued matrices under matrix addition
Set {0, 1} under modulo−2 addition
Refer to REB’s book for more examples

Subgroup
Let G be a group and let H be a subset of G
H is a subgroup of G if H is a group with respect to the restriction of ∗ to H
E.g., Set of integers under addition is a group & set of even integers is a
subgroup
Cyclic group (CG): or Monogenous group
a group that is generated by a single element
Group that consists of all the powers of one of its elements
Every CG of prime order is a simple group which cannot be broken down
into smaller groups
CGs of prime order are among the building blocks from which all groups can
be built
Figure: The six 6th complex roots of unity form a cyclic group under multiplication.
Source: Wiki

Modular Arithmetic
Due to C. F. Gauss
a system of arithmetic for integers, where numbers wrap around upon
reaching a certain value−the modulus
modulo n
For a positive integer n, two numbers a and b are said to be congruent
modulo n, if their difference a − b is an integer multiple of n
i.e. if there is an integer k such that a − b = k × n
e.g., 38 ≡ 14 (mod 12)

Modular Arithmetic
Due to C. F. Gauss
a system of arithmetic for integers, where numbers wrap around upon
reaching a certain value−the modulus
modulo n
For a positive integer n, two numbers a and b are said to be congruent
modulo n, if their difference a − b is an integer multiple of n
i.e. if there is an integer k such that a − b = k × n
e.g., 38 ≡ 14 (mod 12)
A familiar use of modular arithmetic is in the 12-hour clock (Ref. Wiki

webpage)
Figure: Source: Wiki

Coset Decomposition
Example:
Let G is the group, integers mod 8 under addition i.e.

Coset Decomposition
Example:
Let G is the group, integers mod 8 under addition i.e.
G = {0, 1, 2, 3, 4, 5, 6, 7} ⇐ finite
Subgroup H = {0, 4}
Clearly, H ⊂ G
More details in class

Coset Decomposition of G
General construction
Figure: Source: REB’s text
Coset leader: the first element on the left of each row

Coset leaders are used in decoding received codewords in linear block
codes
Left coset: Each row in the array
called simply as coset for abelian group (G, ∗)

Coset Decomposition of G
General construction
Figure: Source: REB’s text
Coset leader: the first element on the left of each row

Coset leaders are used in decoding received codewords in linear block
codes
Left coset: Each row in the array
called simply as coset for abelian group (G, ∗)
Right coset: If the coset decomposition is defined with the elements of G
multiplied on the right, the rows are called right cosets
Coset decomposition is always rectangular with all rows completed
Theorem & Corollary
Theorem
Every element of set G appears once and only once in a coset
decomposition of G
Proof by contradiction
Corollary
If H is a subgroup of G, then the number of elements in H divides the
number of elements in G
(Order of H) × (Number of cosets of G w.r.to H) = (order of G)

Rings
Ring is an abstract set

an abelian group with additional structure

Rings

Definition
A ring R is a set with two operations defined: addition (+) and
multiplication and the following axioms are satisfied:
(R, +) is an abelian group
Closure: For any a, b in R, the product ab is in R
Associativity: a(bc) = (ab)c
Distributive property: a(b + c) = ab + ac, (b + c)a = ba + ca
Multiplication is commutative =⇒

Rings

Definition
A ring R is a set with two operations defined: addition (+) and
multiplication and the following axioms are satisfied:
(R, +) is an abelian group
Closure: For any a, b in R, the product ab is in R
Associativity: a(bc) = (ab)c
Distributive property: a(b + c) = ab + ac, (b + c)a = ba + ca
Multiplication is commutative =⇒ commutative ring

Theorem
Theorem
For any a, b in a ring R,
a0 = 0a = 0 ⇐ identity
a(-b) = (-a)b = -(ab)
ring with identity

Theorem
Theorem
a(-b) = (-a)b = -(ab)
ring with identity

ring R that has an identity under multiplication
1a = a1 = a, for all a in R
Inverse
Given an element a, there may exist an element b with ab = 1 ⇒ right inverse b

Theorem
Theorem
a(-b) = (-a)b = -(ab)
ring with identity

ring R that has an identity under multiplication
1a = a1 = a, for all a in R
Inverse
Given an element a, there may exist an element b with ab = 1 ⇒ right inverse b
If there is an element c with ca = 1 ⇒ left inverse c

Theorems
Theorem
In a ring with identity
The identity is unique
If an element a has both a right inverse b and a left inverse c, then b = c
The inverse is unique, which is a−1
Theorem
Let an element that has inverse is called a unit
Under ring multiplication, the set of units of a ring forms a group
If c = ab and c is a unit, then a has a right inverse and b has left inverse

Examples
The set of real numbers under usual addition and multiplication is

a commutative ring with identity
every nonzero element is a unit
The set of all integers a commutative ring with identity under usual
addition and multiplication is a

Examples

addition and multiplication is a a commutative ring with identity (denoted
by Z)
The only units are

Examples

by Z)
The only units are ±1
The set of all n × n matrices with real valued elements under matrix
addition and matrix multiplication is a

Examples

by Z)
addition and matrix multiplication is a a noncommutative ring with identity
The identity is

Examples

by Z)
The identity is n × n identity matrix, the units are

Examples

by Z)
The identity is n × n identity matrix, the units are the nonsingular matrices
The set of all polynomials in x with real-valued coefficients under
polynomial addition and multiplication is a

Examples

by Z)
polynomial addition and multiplication is a commutative ring with identity
The identity is

Examples

by Z)
polynomial addition and multiplication is a commutative ring with identity
The identity is the zero degree polynomial p(x) = 1

Fields
More powerful algebraic structure
a set in which one can add, subtract, multiply, and divide
Definition
A field F is a set that has two operations defined on it; addition and
multiplication such that the following axioms are satisfied
The set is an abelian group under addition
The field is closed under multiplication, and set of nonzero elements is an
abelian group under multiplication
The distributive property
(a + b)c = ac + bc
for all a, b, c in the field
Examples

Fields
More powerful algebraic structure
a set in which one can add, subtract, multiply, and divide
Definition
A field F is a set that has two operations defined on it; addition and
multiplication such that the following axioms are satisfied
The set is an abelian group under addition
The field is closed under multiplication, and set of nonzero elements is an
abelian group under multiplication
The distributive property
(a + b)c = ac + bc
for all a, b, c in the field
Examples
Q: the set of rational numbers
R: the set of real numbers
C: the set of complex numbers
These fields have infinite number of elements
Galois field (GF)
A field with q elements, if it exists, is called a finite field or a Galois field

denoted by GF (q)
GF (3) = {0, 1, 2}, GF (4) = {0, 1, 2, 3} (details in class)
Note
GF(2) is contained in GF (4) (but not in GF(3))

Subfield
Subfield
Let F be a field. A subset of F is called a subfield, if it is a field under the
inherited addition and multiplication
The original field F is then called an extension field of the subfield
To prove that a subset of finite field F is a subfield

Prove that it contains a nonzero element
It is closed under addition and multiplication
All other necessary properties are inherited from F

Theorem
Theorem
In any field, if ab = ac and a 6= 0, then b = c
Note: Some rings may satisfy the cancellation property, but these rings
are not fields

Theorem
Theorem
are not fields
E.g.,

Theorem
Theorem
are not fields
E.g., the ring of integers, note that a−1 does not exist
Definition
An integral domain is a commutative ring in which b = c whenever
ab = ac and a is nonzero

Vector Spaces
Familiar example

Vector Spaces
Familiar example
three-dimensional (3−D) Euclidean space
can be extended to an n− dimensional vector space over real numbers (Rn )
Vector spaces defined abstractly with respect to any field
Definition
A vector space (V , +, F , .) is a set V of vectors, a field F of scalars, and
two operations vector addition (+) and scalar multiplication (.) such that
(V , +) is an abelian group
c.v ∈ V for all c ∈ F & v ∈ V ⇐

Vector Spaces
Familiar example
Definition
c.v ∈ V for all c ∈ F & v ∈ V ⇐ Closure
c1 (c2 v) = (c1 c2 )v, for all c1 , c2 ∈ F & v ∈ V ⇐

Vector Spaces
Familiar example
Definition
c1 (c2 v) = (c1 c2 )v, for all c1 , c2 ∈ F & v ∈ V ⇐ Associativity
(c1 + c2 )v = c1 v + c2 v, for all c1 , c2 ∈ F & v ∈ V ⇐

Vector Spaces
Familiar example
Definition
c1 (c2 v) = (c1 c2 )v, for all c1 , c2 ∈ F & v ∈ V ⇐ Associativity
(c1 + c2 )v = c1 v + c2 v, for all c1 , c2 ∈ F & v ∈ V ⇐ Distributive property
1.v = v for all v ∈ V ( =⇒ 1 is the multiplication identity in F )
The zero element of V is called the origin of V (denoted by 0)

Examples
(Rn , +, R, .)
(C, +, R, .)

Examples
(Rn , +, R, .)
(C, +, R, .) ≡ (R2 , +, R, .)
(Rm×n , +, R, .)
Set of m × n matrices over reals
Let V be the set of polynomials in x with coefficients in GF (q) and take
F = GF (q)
In this space, vectors are polynomials

Linear Dependence & LI
Definitions
In a vector space V , a sum of the form
u = b1 v1 + b2 v2 + . . . + bn vn ,
where bj , j = 1, 2, . . . , n, are scalars, is called a linear combination of the

vectors v1 , v2 , . . . , vn
Definitions
A set of vectors {v1 , v2 , . . . , vn } is called linearly dependent (LD) if there
is a set of scalars {b1 , b2 , . . . , bn }, not all zero, such that
b1 v1 + b2 v2 + . . . + bn vn = 0.
A set of vectors that is not linearly dependent is called linearly

independent (LI)
No vector in a LI set can be expressed as a linear combination of others

Definitions & Theorem
Definitions
A set of vectors is said to span a vector space if every vector in the space
equals at least one linear combination of the vectors in the set
A vector that is spanned by a finite set of vectors is called a
finite dimensional vector space
Theorem
If a vector space V is spanned by a finite set of m vectors
A = {x1 , x2 , . . . , xm } and V contains a set of n LI vectors
B = {u1 , u2 , . . . , un }, then m ≥ n.

Theorems
Theorem
Two LI sets of vectors that span the same finite-dimensional vector space
have the same number of vectors
The number of LI vectors in a set that spans a finite dimensional vector

space V is called the dimension of V
Theorem
In a n− dimensional vector space V , any set of n LI vectors is a basis for
V

Vector Subspace
A nonempty subset of a vector space is called a vector subspace if

the subset is also a vector space under (+) and (.)
Q. How to check whether a nonempty subset of a vector space is a
subspace
Check for closure under (+) and (.)
Closure under (+) ensures that the zero vector is in the subset
Other properties are inherited form the original vector space
Examples (in class)

Theorems
Theorem
In any vector space V , the set of all linear combinations of a set of
vectors {v1 , v2 , . . . , vn } is a subspace of V
Proof in class
Theorem
If W , a vector space of a finite-dimensional vector space V , has the same
dimension as V , then W = V

Definitions
Definition of n− tuple
Given a field F , the quantity (b1 , b2 , . . . , bn ), composed of field elements
is called n−tuple of elements from the field F
under the operations of componentwise addition and componentwise scalar
multiplication, the set of n− tuples of elements from a field F is a

Definitions
Definition of n− tuple
Given a field F , the quantity (b1 , b2 , . . . , bn ), composed of field elements
is called n−tuple of elements from the field F
under the operations of componentwise addition and componentwise scalar
multiplication, the set of n− tuples of elements from a field F is a vector
space, denoted by F n
Any finite-dimensional vector space can be represented as an n−tuple

space by choosing a basis {e1 , e2 , . . . , en } and representing a vector
v = b1 e1 + b2 e2 + . . . + bn en

Definitions
Definition of inner product

The inner product of two n− tuples of F n
u = (a1 , a2 , . . . , an ), v = (b1 , b2 , . . . , bn )
is

Definitions

u = (a1 , a2 , . . . , an ), v = (b1 , b2 , . . . , bn )
is a scalar defined as
u.v = a1 b1 + a2 b2 + . . . + an bn )
Easy to verify that
u.v = v.u, (cu).v = c(u.v), w.(u + v) = (w.u) + w.v
If the inner product (or dot product) of two vectors is zero, they are said to
be

Definitions

u = (a1 , a2 , . . . , an ), v = (b1 , b2 , . . . , bn )
is a scalar defined as
u.v = a1 b1 + a2 b2 + . . . + an bn )
Easy to verify that
u.v = v.u, (cu).v = c(u.v), w.(u + v) = (w.u) + w.v
If the inner product (or dot product) of two vectors is zero, they are said to
be orthogonal

Theorem
Theorem
Let V be the vector space of n−tuples over a field F , and let W be a
subspace. The set of vectors orthogonal to W is itself a subspace

References & Further Reading
Theory and practice of error control codes by Richard E Blahut

Wikipedia

Slides On Algebra of Coding Theory

Uploaded by

Slides On Algebra of Coding Theory

Uploaded by

Algebra for Coding Theory

Department of Electrical and Electronics Engineering

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 1 / 30

1 The Binary Field

7 References & Further Reading

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 2 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 3 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 3 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 4 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 5 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 5 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 5 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 5 / 30

Order of G: number of elements in G

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 5 / 30

Order of G: number of elements in G

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 5 / 30

Order of G: number of elements in G

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 5 / 30

Refer to REB’s book for more examples

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 6 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 7 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 8 / 30

A familiar use of modular arithmetic is in the 12-hour clock (Ref. Wiki

Figure: Source: Wiki

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 8 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 9 / 30

Figure: Source: Wiki

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 9 / 30

Figure: Source: REB’s text

Coset leader: the first element on the left of each row

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 10 / 30

Figure: Source: REB’s text

Coset leader: the first element on the left of each row

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 11 / 30

Ring is an abstract set

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 12 / 30

Ring is an abstract set

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 12 / 30

Ring is an abstract set

Multiplication is commutative =⇒ commutative ring

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 12 / 30

ring with identity

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 13 / 30

ring with identity

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 13 / 30

ring with identity

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 13 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 14 / 30

The set of real numbers under usual addition and multiplication is

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 15 / 30

The set of real numbers under usual addition and multiplication is

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 15 / 30

The set of real numbers under usual addition and multiplication is

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 15 / 30

The set of real numbers under usual addition and multiplication is

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 15 / 30

The set of real numbers under usual addition and multiplication is

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 15 / 30

The set of real numbers under usual addition and multiplication is

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 15 / 30

The set of real numbers under usual addition and multiplication is

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 15 / 30

The set of real numbers under usual addition and multiplication is

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 15 / 30

for all a, b, c in the field

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 16 / 30

for all a, b, c in the field

A field with q elements, if it exists, is called a finite field or a Galois field

Figure: Source: Wiki

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 17 / 30

To prove that a subset of finite field F is a subfield

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 18 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 19 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 19 / 30

B. Sainath (BITS, PILANI) Algebra & Linear Algebra September 7, 2019 19 / 30