Let us introduce notation
\begin{equation*}
\gamma (0,1)=\gamma _{1}, \quad
\gamma (0,2)=\gamma _{2}, \quad
\gamma (1,2)=\gamma , \quad
u_{0}^{\mu }=U^{\mu}.
\end{equation*}
Each particle's four velocity can be decomposed into components parallel and orthogonal to $U^\mu$ as follows
\begin{equation}
u_{1}^{\mu }=a_{1}U^{\mu }+b_1 n_{1}^{\mu}, \label{ab}
\end{equation}
where $n^{\mu }$ is a unit space-like vector orthogonal to $U^{\mu }$. It is easy to
find from (\ref{ab}) that
\begin{equation}
a_{1}=-u_{1}^{\mu }U_{\mu }=\gamma _{1},
\end{equation}
and from the normalization conditions for $u_{1}^{\mu }$ and $U^{\mu }$ then we see that $b_1$ as introduced in (\ref{ab}) is the velocity of particle 1 in the frame of particle 0:
\begin{equation}
b_{1}^{2}=\frac{1}{\gamma _{1}^{2}-1 },
\end{equation}
so in terms of velocity of particle 1 in the frame of particle 0
\begin{equation}
w_1 =(1-\gamma_1^{-2})^{1/2}=\frac{b_1}{\gamma},
\end{equation}
the decomposition takes form
\begin{equation}
u_{1}^{\mu }=\gamma_{1}(U^{\mu }+w_1 n_{1}^{\mu}).
\end{equation}
Writing down for particle 2 the equation analogous to (\ref{ab}) and
calculating the scalar product $u_{1}^{\mu }u_{2\mu }$, one obtains
\begin{align}
\gamma &=\gamma _{1}\gamma _{2}
-b_1 b_2 (n_{1}^{\mu }n_{2\mu})\\
&=\gamma _{1}\gamma _{2}\big(1-\varepsilon\; w_{1}w_{2}\big),
\label{ga}
\end{align}
where
\begin{equation}
\varepsilon =(n_{1}^{\mu }n_{2\mu }),\qquad |\varepsilon|\leq 1 ,
\end{equation}
or in terms of velocities
\begin{equation}
\frac{1}{\sqrt{1-w^{2}}}
=\frac{1}{\sqrt{1-w_{1}^{2}}}
\frac{1}{\sqrt{1-w_{2}^{2}}}
(1-\varepsilon w_{1}w_{2}). \label{gw}
\end{equation}
In case particle 0 is at rest in the laboratory frame $U^{\mu }=(1,0)$, we have $n_{a}^{\mu}=(0,\vec{n}_{a})$, $a=1,2$, and
\begin{equation}
u_{a}^{\mu }=\gamma _{a}(1,w_{a}\vec{n}_{a}),\qquad \varepsilon =(\vec{n}_{1}
\vec{n}_{2}).
\end{equation}
Then, eqs. (\ref{ga}), (\ref{gw}) turn into those listed in problem 1.3 of Lightman et al $^*$.
$^*$ A. P. Lightman, W. H. Press, R. H. Price, and S. A. Teukolsky, Problem book in Relativity and Gravitation (Princeton University Press, Princeton, New Jersey, 1975); ISBN 069-108-160-3.