We summarize some product operations in vector / matrix / tensor / group. A recurring theme: many of these products are instances of universal constructions in category theory (when you have a category hammer, everything is a category nail).

Vector

Inner Product / Dot Product

For two vectors $\mathbf{u}$ and $\mathbf{v}$, and they must have same dimension $n$, $\mathbf{u}, \mathbf{v} \in \mathbb{R}^{n}$ Inner product denoted as $\mathbf{u} \cdot \mathbf{v}$ or $\langle\mathbf{u}, \mathbf{v}\rangle$ Result is a scalar $\mathbb{R}^{n} \times \mathbb{R}^{n} \rightarrow \mathbb{R}$ Calculation $\mathbf{u} \cdot \mathbf{v}=\sum_i \mathbf{u}_i \mathbf{v}_i$ Commutative: $\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$

Outer Product

For two vectors $\mathbf{u}$ and $\mathbf{v}$, but they don’t need to have same dimension $\mathbf{u}\in \mathbb{R}^{m}$, $\mathbf{v}\in \mathbb{R}^{n}$ Result is a matrix $\mathbf{u} \otimes \mathbf{v}=\mathbf{u} \mathbf{v}^{\top} \in \mathbb{R}^{m \times n}$ The inner product equals the trace of the outer product: $\mathbf{u} \cdot \mathbf{v} = \operatorname{Tr}(\mathbf{u} \mathbf{v}^T)$. Not commutative.

Cross Product

Usually in $\mathbf{u}, \mathbf{v} \in \mathbb{R}^{3}$ Result is a vector perpendicular to both : $\mathbb{R}^3 \times \mathbb{R}^3 \rightarrow \mathbb{R}^3$, direction follows the right-hand rule Calculation: $\mathbf{u} \times \mathbf{v}=\lVert \mathbf{u}\rVert\lVert \mathbf{v}\rVert \sin (\theta) \mathbf{n}$ where $\mathbf{n}$ is a unit vector perpendicular to the plane containing $\mathbf{u}$ and $\mathbf{v}$, Anti-commutative: $\mathbf{u} \times \mathbf{v}=-\mathbf{v} \times \mathbf{u}$

Matrix

Hadamard Product / Element-wise product

For two matrices $A$ and $B$ of the same dimension $m \times n$, the Hadamard product $A \odot B$ is a matrix of the same dimension as the operands, with elements given by

\[(A \odot B)_{i j}=(A)_{i j}(B)_{i j} .\]

Result is a matrix of same size: $\mathbb{R}^{m \times n} \times \mathbb{R}^{m \times n} \rightarrow \mathbb{R}^{m \times n}$

Kronecker Product

If $\mathbf{A}$ is an $m \times n$ matrix and $\mathbf{B}$ is a $p \times q$ matrix, then the Kronecker product $\mathbf{A} \otimes \mathbf{B}$ is the $p m \times q n$ block matrix:

\[\mathbf{A} \otimes \mathbf{B}=\begin{bmatrix} a_{11} \mathbf{B} & \cdots & a_{1 n} \mathbf{B} \\ \vdots & \ddots & \vdots \\ a_{m 1} \mathbf{B} & \cdots & a_{m n} \mathbf{B} \end{bmatrix},\]

It is a specialization of the tensor product.

Matrix Multiplication

If $\mathbf{A}$ is an $m \times n$ matrix and $\mathbf{B}$ is an $n \times p$ matrix, the matrix product $\mathbf{C}=\mathbf{A B}$ is defined to be the $m \times p$ matrix Denoted without multiplication signs or dots Special case in vector: inner product and outer product

Group

Binary Operation (Same Group)

Sometimes also called ‘group product’ to distinguish from ‘group action’, which might be confusing. A group $G$ has a binary operation

\[*: G \times G \rightarrow G, \quad(x, y) \mapsto x * y\]

(usually written $x y$ ). It’s only defined when both inputs are in the same group. The output is again in $G$. The symbol “ $\times$ “ is the Cartesian product of sets. Unfortunately, the same symbol $\times$ is used for the direct product.

Group Action (Between Two Groups)

An action of $H$ on $N$ is a map

\[\cdot: H \times N \rightarrow N, \quad(h, n) \mapsto h \cdot n\]

such that each $h$ acts by an automorphism of $N$ and

\[e \cdot n=n, \quad\left(h_1 h_2\right) \cdot n=h_1 \cdot\left(h_2 \cdot n\right) .\]

Equivalently, a homomorphism $\varphi: H \rightarrow \operatorname{Aut}(N)$ with $h \cdot n:=\varphi(h)(n)$. Here the inputs come from different groups, and the output lies in $N$.

Direct Product (Cartesian Product)

Denoted as $G\times H$ or $G \oplus H$. Result is a group. Given groups $G$ (with operation $*$ ) and $H$ (with operation $\Delta$ ), the direct product $G \times H$ is defined as follows:

1. The underlying set is the Cartesian product, $G \times H$. That is, the ordered pairs $(g, h)$, where $g \in G$ and $h \in H$.
2. The binary operation on $G \times H$ is defined component-wise:

\[\left(g_1, h_1\right) \cdot\left(g_2, h_2\right)=\left(g_1 * g_2, h_1 \Delta h_2\right)\]

The resulting algebraic object satisfies the axioms for a group.

In direct product $G\times H$, we naturally have : $g\in G$, $g_1* g_2\in G$ $h\in H$, $h_1\Delta h_2\in H$ $(g,h)\in G\times H$, $(g_1, h_1)\cdot (g_2,h_2)\in G\times H$ But $\left(g_1, h_1\right) \cdot\left(g_2, h_2\right)=\left(g_1 * g_2, h_1 \Delta h_2\right)$ is a new relation

Universal property

The direct product $G \times H$ can be characterized by the following universal property. Let $\pi_G: G \times H \rightarrow G$ and $\pi_H: G \times H \rightarrow H$ be the projection homomorphisms. Then for any group $P$ and any homomorphisms $f_G: P \rightarrow G$ and $f_H: P \rightarrow H$, there exists a unique homomorphism $f: P \rightarrow G \times H$ making the following diagram commute:

Specifically, the homomorphism $f$ is given by the formula

\[f(p)=\left(f_G(p), f_H(p)\right)\]

$(g, e_H)$ and $(e_G, h)$ commute inside $G \times H$ by definition. The two coordinates are ‘independent’. Two embedded subgroups centralize each other. That is, $G \times \lbrace e_H\rbrace$ and $\lbrace e_G\rbrace \times H$ centralize each other. But $G \times H$ need not to be abelian, $G \times H$ is abelian iff both $G$ and $H$ are abelian.

Free Product

Elements are reduced words alternating $G$-letters and $H$-letters. For example, in $\mathbb{Z} * \mathbb{Z} = \langle a, b \rangle$, the element $aba^{-1}b^2$ is a reduced word. In $\mathbb{Z} \times \mathbb{Z}$, this would simplify to $b^3$ (since $a$ and $b$ commute) – but in the free product, it does not. No relations are imposed between $G$-letters and $H$-letters.

Universal property

Similar to direct product but inverse arrows.

But the flip in arrow direction leads to genuinely different algebraic behavior. $GH$ gives no new relation to the mixed. $G * H$ does not commute internally. If $GH$ commutes, then we should be able to find a homomorphic $K$ which also commutes, but in fact we can choose arbitrary $K$ which not commutes.

Example $\mathbb{Z} \times \mathbb{Z}=\langle a, b \mid a b=b a\rangle\cong\pi_1(S^1\times S^1)$ which is a torus. $\mathbb{Z} * \mathbb{Z}=\langle a, b \mid\rangle\cong\pi_1(S^1\vee S^1)$ which is figure 8.

Semidirect Product

Internal semidirect product

A group $G$ is an internal semidirect product of subgroups $N, H$ (write $G=N \rtimes H$ ) if:

1. $N \triangleleft G$,
2. $G=N H$, $g=n h$
3. $N \cap H=\lbrace e\rbrace$

Then conjugation by $H$ on $N$ gives the action $\varphi_h: N \rightarrow N$, $\varphi_h(n)=h n h^{-1}$, and the map

\[N \rtimes_{\varphi} H \xrightarrow{\cong} G, \quad(n, h) \mapsto n h\]

is a group isomorphism.

Condition 2 3 make sure $N\times H\rightarrow G$ is bijective (here $\times$ is Cartesian product not direct group product) Condition 1 $hnh^{-1}\in N$ gives $N\times H$ group structure, so isomorphism Group law in $G$:

\[\left(n_1, h_1\right) \cdot\left(n_2, h_2\right):=\left(n_1 \varphi_{h_1}\left(n_2\right), h_1 h_2\right)=(n_1h_1n_2h_1^{-1},h_1h_2)=n_1h_1n_2h_2\]

Since $N \triangleleft G$, the conjugation by $H$ preserves $N$ :

\[h n h^{-1} \in N\]

So we automatically get a homomorphism

\[\varphi: H \rightarrow \operatorname{Aut}(N), \quad \varphi_{h}(n)=h n h^{-1} .\]

$\varphi$ is forced by conjugation of $H$ on $N$ Remark: $\varphi_{h}(n)=h n h^{-1}$ is an action of $h$ on $n$.

External semidirect product

Given groups $N, H$ and choose a homomorphism

\[\varphi_h: H \longrightarrow \operatorname{Aut}(N),\]

the semidirect product $N \rtimes_{\varphi} H$ is the set $N \times H$ with multiplication

\[\boxed{\left(n_1, h_1\right)\cdot\left(n_2, h_2\right)=\left(n_1 \varphi_{h_1}\left(n_2\right), h_1 h_2\right)}\]

where $\varphi_h(n)\in N$ is an action (not product) of $h$ on $n$ by an automorphism of $N$ . (Internal semidirect product this action is conjugation).

In $N \rtimes_{\varphi} H$

• Identity: $\left(e_N, e_H\right)$.
• Inverse:

\[(n, h)^{-1}=\left(\varphi\left(h^{-1}\right)\left(n^{-1}\right), h^{-1}\right) .\]

• Conjugation of $N$ by $H$ :

\[(e, h)(n, e)(e, h)^{-1}=(\varphi_{h}(n), e) .\]

$\varphi$ is chosen, it prescribes how $H$ twists $N$ inside the product. The trivial choice always exists. Intuitively, in $N \rtimes_{\varphi} H$ , the second group $H$ ‘twist’ the elements in first group $N$, and the result is a group whose normal subgroup is $N$.

Direct product as a special case.

If the action is trivial ( $\varphi(h)=\operatorname{id}_N$ for all $h$ ), then the law become

\[\left(n_1, h_1\right)\left(n_2, h_2\right)=\left(n_1 n_2, h_1 h_2\right)\] \[N \rtimes_{\text {triv }} H=N \times H\]

Category

Category product

Category Product

Tensor Product

The tensor product $\otimes$ is part of the structure of a monoidal category. For simplicity, we take the vector space for example) Given two vector spaces $V$ and $W$, their tensor product $V \otimes W$ is another vector space with $\dim(V \otimes W) = \dim(V) \cdot \dim(W)$. (Compare with the Cartesian product $V \times W$, which has dimension $\dim(V) + \dim(W)$.) Definition using universal property: For vector spaces $V, W$ over $F$, the tensor product $V \otimes W$ is a vector space together with a bilinear map

\[\varphi: V \times W \longrightarrow V \otimes W, \quad(v, w) \mapsto v \otimes w,\]

such that for every vector space $Z$ and every bilinear map $h: V \times W \rightarrow Z$, there exists a unique linear map $\tilde{h}: V \otimes W \rightarrow Z$ with $\tilde{h}(v \otimes w)=h(v, w)$, i.e. the diagram

commutes. Equivalently:

\[\operatorname{Lin}(V \otimes W, Z) \cong \operatorname{Bilin}(V \times W, Z) \quad \text { naturally in } Z .\]

Intuitively(?), $V \otimes W$ is the ‘most general’ vector space from a bilinear map $\varphi$, that can uniquely linearly (because of vector space) factor (not generate) any other vector space $Z$ generated by any bilinear map $h$. And there exists a unique linear factorization $\tilde{h}$ of $h$. Categorically, $(V \otimes W, \varphi)$ is initial in the category whose objects are bilinear maps out of $V \times W$. Any other such pair $(Z, h)$ receives a unique linear map $V \otimes W \rightarrow Z$ taking $\varphi$ to $h$. Equivalently, there is a natural isomorphism of functors $Z \mapsto \operatorname{Lin}(V\otimes W, Z)$ and $Z \mapsto \operatorname{Bilin}(V \times W, Z)$. $V \otimes W$ is universal for bilinear maps out of $V \times W$ : every such map $h$ factors uniquely through $\varphi$ by a linear $\tilde{h}$.

Defining a new category might be a bit clearer: Fix vector spaces $V, W$ over a field $F$. Consider the category $\mathcal{C}$ whose

• objects are pairs $(T, h)$ with a bilinear map $h: V \times W \rightarrow T$,
• morphisms $(T, h) \rightarrow\left(T^{\prime}, h^{\prime}\right)$ are linear maps $u: T \rightarrow T^{\prime}$ satisfying $u \circ h=h^{\prime}$. A tensor product of $V$ and $W$ is a pair $(V \otimes W, \varphi)$ with $\varphi: V \times W \rightarrow V \otimes W$ bilinear such that: For every $(T, h)$ in $\mathcal{C}$ there exists a unique linear map $\tilde{h}: V \otimes W \rightarrow T$ with $\tilde{h} \circ \varphi=h$. Equivalently:

\[\operatorname{Lin}(V \otimes W, Z) \cong \operatorname{Bilin}(V \times W, Z) \quad \text { naturally in } Z,\]

The abstract tensor product is basis-free, once we pick bases and turn maps into matrices, the tensor product becomes the Kronecker product.