%\documentclass[12pt]{article}
\documentclass{article}
\usepackage{amsmath,amsthm,amssymb}
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
 \usepackage{graphicx}
 \usepackage{listings}
 \usepackage{xcolor}
\usepackage{fancyhdr}
\usepackage{hyperref}
\graphicspath{{figures/}}

\pagestyle{fancy}
\fancyhf{}
\rhead{David Dobor and Longin Jan Latecki}
\lhead{Google's PageRank}
\rfoot{Page \thepage}


\definecolor{mygreen}{rgb}{0,0.6,0}
\definecolor{mygray}{rgb}{0.5,0.5,0.5}
\definecolor{mymauve}{rgb}{0.58,0,0.82}


\begin{document}
\lstset{ %
  backgroundcolor=\color{white},   % choose the background color; you must add \usepackage{color} or \usepackage{xcolor}
  basicstyle=\footnotesize,        % the size of the fonts that are used for the code
  breakatwhitespace=false,         % sets if automatic breaks should only happen at whitespace
  breaklines=true,                 % sets automatic line breaking
  captionpos=b,                    % sets the caption-position to bottom
  commentstyle=\color{mygreen},    % comment style
  deletekeywords={...},            % if you want to delete keywords from the given language
  escapeinside={\%*}{*)},          % if you want to add LaTeX within your code
  extendedchars=true,              % lets you use non-ASCII characters; for 8-bits encodings only, does not work with UTF-8
  %frame=single,                    % adds a frame around the code
  keepspaces=true,                 % keeps spaces in text, useful for keeping indentation of code (possibly needs columns=flexible)
  keywordstyle=\color{blue},       % keyword style
  language=Matlab,                 % the language of the code
  morekeywords={*,...},            % if you want to add more keywords to the set
  numbers=none,                    % where to put the line-numbers; possible values are (none, left, right)
  numbersep=5pt,                   % how far the line-numbers are from the code
  numberstyle=\tiny\color{mygray}, % the style that is used for the line-numbers
  rulecolor=\color{black},         % if not set, the frame-color may be changed on line-breaks within not-black text (e.g. comments (green here))
  showspaces=false,                % show spaces everywhere adding particular underscores; it overrides 'showstringspaces'
  showstringspaces=false,          % underline spaces within strings only
  showtabs=false,                  % show tabs within strings adding particular underscores
  %stepnumber=2,                    % the step between two line-numbers. If it's 1, each line will be numbered
  stringstyle=\color{mymauve},     % string literal style
  tabsize=2,                       % sets default tabsize to 2 spaces
  title=\lstname                   % show the filename of files included with \lstinputlisting; also try caption instead of title
}
\begin{center}
\section*{A Note on Google's PageRank}
\end{center}

\renewcommand{\abstractname}{ }
\begin{abstract}
According to Google, google-search on a given topic results in a listing of most relevant web pages related to the topic.  Google ranks the importance of webpages according to an eigenvector of a weighted link matrix.  The following offers an insight into how this is done and is a basic application of the eigenvalue problem from linear algebra. 
It is based on the \emph{Bryan, Leise paper}
and on
\url{http://www.math.cornell.edu/~mec/Winter2009/RalucaRemus/Lecture3/lecture3.html}.


\end{abstract}

\subsection*{A Tiny Web Example}


\begin{itemize}
\item  Core idea: in assigning a score to any given web page, the page's score (ranking)
 is derived from the links made \emph{to} that page from other web pages.

\item  The links \emph{to} a given page are called the \emph{backlinks} for that page

\item  The web is represented as a directed graph $G = (V,E)$ with vertices
 being the web pages and edges the links.  There is a directed edge from page
$i$ to page $j$ if page $i$ contains a hyperlink to page $j$.

\item  Denote the importance score of page $k$ by $x_k$ ($x_i > x_j$ means
  that page $i$ is more important)

\item  As in the paper, the approach that \emph{doesn't work} is to take $x_k$ as
 the number of backlinks for page $k$, 
 e.g. here $x_1 = 2, x_2 = 1, x_3 = 3, x_4 = 2$:

\begin{center}
\end{center}
\begin{figure*}[htb]
	\begin{center}
		\includegraphics[width=0.32\linewidth]{graf1} \hspace{3mm}
		\includegraphics[width=0.3\linewidth]{graf2} \hspace{3mm}
		\includegraphics[width=0.25\linewidth]{matrix}
		%\includegraphics{ExampleGraph1}
	\end{center}
	\caption{An example web graph, its weighted version, and its link matrix (transpose of the adjacency matrix of the weighted graph).}
	\label{fig:graph1}
\end{figure*}

But we want a link to page $k$ from an important page to boost page $k$’s
 importance score more than a link from an unimportant page. 
 (A page's importance is presumably higher when, say, the CNN's webpage links to it than when just Joe Blow's web page links to it.)

 E.g., in the above graph, pages 1 and 4 have the same score, but one of page 1’s backlinks is
 from the seemingly important page 3 (which seems important because everybody else links to it), while one of page 4’s
 backlinks is from the relatively unimportant page 1. Thus, we'd be rating page 1’s  importance higher than page 4's.

 \item  In an attempt to fix this, we can try to compute the score of page $j$ as the \emph{sum} of the
 scores of all pages linking to page $j$.

For example, the score of page 1 would be determined by the relation
 $x_1 = x_3 +x_4$, because pages 3 and 4 are 1's backlilnks and their
 scores are $x_3$ and $x_4$.

 \item  However, there's a bit of a problem with this:  we don’t want a single individual webpage to
 gain influence merely by casting multiple votes  (just as in elections, we don't want a single individual to gain undue influence by casting multiple votes)

  \item So we make a correction: if page $j$ contains $n_j$ \emph{out}links, one of which links to page $k$, then we boost page $k$'s score by $x_j / n_j$ rather than by $x_j$.

\item Notice that in this scheme each web page gets a total of one
 vote, weighted by that web page’s score, that is evenly divided up among
 all of its outgoing links.

 Let $L_k \subseteq \{1, 2, \dots, n \}$ denote the set of pages with a link to page $k$, that is,
$L_k$ is the set of $k$'s backlinks.  For each $k$ require:

                 \begin{equation} x_k = \sum_{j \in L_k} \frac{x_j}{n_j} \end{equation}

where $n_j$ is the set of outgoing links from page $j$.

\end{itemize}



\subsection*{Assigning a Score to a Page, an Example}

\begin{itemize}
\item \textbf{Linear algebra point of view:}
For the web in the above figure, using the outlined scheme, we have:
\begin{align*}
        x_1 &= x_3/1 + x_4 / 2 \\
       x_2 &= x_1 / 3 \\
       x_3 & = x_1 / 3 + x_2 / 2 + x_4 / 2\\
       x_4  &= x_1/3 + x_2/2\\
\end{align*}

\item These linear equations can be written as $\mathbf{A}\mathbf{x} = \mathbf{x}$:
$$
\begin{bmatrix}
       0     &  0   &  1 & 1/2 \\
      1/3  & 0    & 0 & 0 \\
      1/3  & 1/2 & 0 & 1/2\\
      1/3 &  1/2 &  0 &  0 \\
 \end{bmatrix}
 \begin{bmatrix}
       x_1 \\
      x_2 \\
      x_3\\
      x_4\\
 \end{bmatrix}
 =
  \begin{bmatrix}
       x_1 \\
      x_2 \\
      x_3\\
      x_4\\
 \end{bmatrix}
$$

\item \emph{Thus, we have reduced the web ranking problem to the problem of finding an eigenvector for the link matrix $\mathbf{A}$: $\mathbf{A} \mathbf{x} = \lambda \mathbf{x}$. In particular, we are looking for the eigenvector corresponding to the eigenvalue $\lambda = 1$.}
(Note that $\mathbf{A}$ is not the graph adjacency matrix. 
$\mathbf{A}^T$ is the graph adjacency matrix.)

\item \textbf{Definition:} The \emph{link matrix} $\mathbf{A}$ is such that $A_{ij} = 1 / n_j$ if page $j$ links to page $i$ and $0$ otherwise\footnote{The number of non-zero entries in the $i$th row of $\mathbf{A}$ is the \emph{in-degree} of node $i$ - i.e. how many other pages link to it. And the number of non-zero entries in the $j$th column is the \emph{out-degree} of node $j$ - i.e. how many other pages $j$ links to.}.  

\item \textbf{Definition:} A matrix $\mathbf{B}$ is called \emph{column-stochastic}
if the sum of entries of each column of $\mathbf{B}$ is one.

\item If the web graph has no `dangling' nodes (a dangling webpage is a page that has no outgoing links), e.g., see Fig.~\ref{dangnode}, then the $j$-th column of $\mathbf{A}$ contains $n_j$ non-zero entries summing up to 1 (each is $1/n_j$). So $\mathbf{A}$'s columns all sum up to 1 and hence $\mathbf{A}$ is column-stochastic.
This implies that $A$ always has 1 as its eigenvalue as we will show now:

\textbf{Claim:} Every column-stochastic matrix has 1 as an eigenvalue. 

\textbf{Proof}: Let $\mathbf{e}$ be a vector of all ones. Obviously $\mathbf{A}^T \mathbf{e} = \mathbf{e}$  holds, since the rows of $\mathbf{A}^T$ add up to one. Thus 1 is an eigenvalue of $\mathbf{A}^T$. Recalling that the eigenvlues of $\mathbf{A}^T$ and $\mathbf{A}$ are the same, proves the claim.
 

\item In this small example the eigenvector corresponding to $\lambda = 1$ is easy to find `by hand'. The following \texttt{MATLAB} code also finds the eigenvalus and eigenvectors of $\mathbf{A}$:
\begin{lstlisting}
A = [  0   0   1  1/2;
      1/3  0   0   0 ;
      1/3 1/2  0  1/2;
      1/3 1/2  0   0 ];

[V D] = eig(A);

x = V(:,1)/sum(V(:,1))  % scale 1'st eigenvector to sum to 1
\end{lstlisting}
We obtain $x=\frac{1}{31}[12 \ 4 \ 9 \ 6] \approx [0.38 \ 0.12 \ 0.29 \ 0.19]$.
We call $x$ the PageRank vector of our web graph.
Note that its components sum up to 1. 
We can plot the rankings as well:
\begin{center}
\includegraphics[scale=.2]{ranking_bar}
\end{center}

\item \textbf{Dynamical systems point of view:}
Suppose that initially the importance is uniformly distributed among the 4 nodes, each getting $1/4$. Denote by $\mathbf{v}$ the initial rank vector, having all entries equal to $1/4$. Each incoming link increases the importance of a web page, so at step 1, we update the rank of each page by adding to the current value the importance of the incoming links. This is the same as multiplying the matrix $\mathbf{A}$ with $\mathbf{v}$. 
At step 1, the new importance vector is $\mathbf{v}_1 = \mathbf{A}\mathbf{v}$. We can iterate the process, thus at step 2, the updated importance vector is 
$\mathbf{v}_2 = \mathbf{A}(\mathbf{A}\mathbf{v}) = \mathbf{A}^2 \mathbf{v}$. 
Numeric computations give:
\begin{center}
	\includegraphics[scale=.92]{rezultate_1}
\end{center}
We notice that the sequences of iterates 
$\mathbf{v}, \mathbf{A}\mathbf{v}, ..., \mathbf{A}^k\mathbf{v}$ tend 
to the equilibrium value $\mathbf{v}^*$, which in this case is 
$\mathbf{v}^* \approx \mathbf{A}^8\mathbf{v}$.
We observe that $\mathbf{v}^*$ is equal to the eigenvector $\mathbf{x}$ computed above.

\item \textbf{Probabilistic point of view:}
Since the importance of a web page is measured by its popularity (how many incoming links it has), we can view the importance of page $i$ as the probability that a random surfer on the Internet that opens a browser to any page and starts following hyperlinks, visits the page i. We can interpret the weights we assigned to the edges of the graph in a probabilistic way: A random surfer that is currently viewing web page 2, has 1/2 probability to go to page 3, and 1/2 probability to go to page 4. We can model the process as a random walk on graphs. Each page has equal probability 1/4 to be chosen as a starting point. So, the initial probability distribution is given by the column vector 
$\mathbf{x}=[\frac{1}{4} \ \frac{1}{4} \ \frac{1}{4} \ \frac{1}{4} ]^T$. 
The probability that page $i$ will be visited after one step is equal to $\mathbf{A}\mathbf{x}$, and so on. 
The probability that page $i$ will be visited after $k$ steps is equal to $\mathbf{A}^k\mathbf{x}$. 
The sequence $\mathbf{A}\mathbf{x}, \mathbf{A}^2\mathbf{x}, \mathbf{A}^3\mathbf{x}, ..., \mathbf{A}^k\mathbf{x}, ...$ converges in this case to a unique probabilistic vector $\mathbf{v}^*$. In this context $\mathbf{v}^*$ is called the stationary distribution. Moreover, the $i$th entry in $\mathbf{v}^*$ is simply the probability that at each moment a random surfer visits page $i$. The computations are identical to the ones we did in the dynamical systems interpretation, only the meaning we attribute to each step being slightly different.

\end{itemize}



\subsection*{Dangling Nodes}

\begin{itemize}
\item If the web graph has dangling nodes, they can be easily repaired. 
A column in the link matrix that represents a dangling node
has all zero entries, since a dangling node has no outgoing edges.
We can replace each such column with a column vector with all entries $1/n$,
where $n$ is the total number of graph nodes, see Fig.~\ref{dangnode}. In this way, the importance of the node is equally redistributed among the other nodes of the graph, instead of being lost.
Since it is equally redistributed, the relative ranking of the other nodes did not change.
For example, we can fix the dangling node E in Fig.~\ref{dangnode} by replacing the column corresponding to E with a column vector with all entries 1/5. 

\begin{figure}[h]
	\begin{center}
		\includegraphics[width=0.5\linewidth]{dangnode.png}
	\end{center}
	\caption {Example of a dangling node.}
	\label{dangnode}
\end{figure}

\end{itemize}


\subsection*{Case of Disconnectd SubWebs}

\begin{itemize}
\item There maybe more than one eigenvector that solves 
$\mathbf{A} \mathbf{x} =  \mathbf{x}$ if the web graph is not connected.
In this case the solution is not unique, 
and hence the dimensionality of the eigenspace 
$V_1(\mathbf{A})$ corresponding to $\lambda =1$ is larger that one.
For example, there are two eigenvectors that solve 
$\mathbf{A} \mathbf{x} =  \mathbf{x}$ in Fig.~\ref{disgraph}.

\begin{figure}[h]
	\begin{center}
		\includegraphics[width=0.45\linewidth]{componente.png} \hspace{3mm}
		\includegraphics[width=0.2\linewidth]{AofDG.png} \hspace{3mm}
		\includegraphics[width=0.2\linewidth]{uv.png}		
	\end{center}
	\caption {Example of a disconnected web graph, its link matrix with two different eigenvectors corresponding to eigenvalue 1.}
	\label{disgraph}
\end{figure}

\end{itemize}

\newpage
\begin{itemize}
\item To solve this problem Page and Brin replaced the matrix $\mathbf{A}$ with 
the matrix: 
$$\mathbf{M} = (1 - m) \mathbf{A} + m \mathbf{S},$$ 
where $\mathbf{S}$ is $n \times n$ matrix with all entries 1/n.
This means we add 'weak' links form every webpage to every other.
The value of $m$ originally used by Google
is reportedly 0.15. For any $m \in [0, 1]$ the matrix $\mathbf{M}$ is column-stochastic and we can show
that $V_1(\mathbf{M})$ is always one-dimensional if $m \in (0, 1]$
if there is no dangling nodes.

\item Observe that: 
$$\mathbf{M} \mathbf{x} = (1 - m) \mathbf{A} \mathbf{x} + m \mathbf{S} \mathbf{x} 
= (1 - m) \mathbf{A} \mathbf{x} + m \mathbf{s},$$
where $\mathbf{s}$ is a column vector with all entries 1/n, since
$\mathbf{S} \mathbf{x} = \mathbf{s}$ for any vector $\mathbf{x} =[x_1, \dots, x_n]^T$
such that $\sum_{i=1}^n x_i = 1$.

\item Hence, the equation $\mathbf{x} = \mathbf{M} \mathbf{x}$ can also be cast as \ \ 
$\mathbf{x} = (1 - m) \mathbf{A} \mathbf{x} + m \mathbf{s}.$


\end{itemize}

\subsection*{PageRank Foundations}

\begin{itemize}
\item By the following theorem, 
matrix $\mathbf{M}$ has a unique eigenvector, corresponding to the eigenvalue 1,
which is called 
the PageRank vector for the web graph with link matrix $\mathbf{A}$.

\textbf{Perron-Frobenius Theorem:} If $\mathbf{M}$ is a positive, column stochastic matrix, then
\begin{enumerate}
\item 1 is an eigenvalue of multiplicity one,
\item 1 is the largest eigenvalue: all the other eigenvalues have absolute value smaller than 1, and
\item the eigenvectors corresponding to the eigenvalue 1 have either only positive entries or only negative entries. In particular, for the eigenvalue 1 there exists a unique eigenvector with the sum of its entries equal to 1.
\end{enumerate}

\item 
From the mathematical point of view, once we have M, computing the eigenvector corresponding to the eigenvalue 1 is, at least in theory, a straightforward task. 
But when the matrix $\mathbf{M}$ has size 30 billion (as it does for the real Web graph), even mathematical software such as Matlab or Mathematica are clearly overwhelmed.
An alternative way of computing the eigenvector is given by the Power Method. 
The theorem below guarantees that the method works for positive, column stochastic matrices.

\textbf{Power Method Convergence Theorem:} 
Let $\mathbf{M}$ be a positive, column stochastic $n \times n$ matrix. 
Denote by $\mathbf{v}^*$ its eigenvector corresponding to the eigenvalue 1. 
Let $\mathbf{v}$ be the column vector with all entries equal to  1/n. 
Then the sequence $\mathbf{v}, \mathbf{M}\mathbf{v}, ..., \mathbf{M}^k\mathbf{v}$ converges to vector $\mathbf{v}^*$.

\item In the random surfer model,
we argued that the iteration process corresponds to the way importance distributes over the net following the link structure. 
Computationally speaking, it is much more easier, starting from the vector with $\mathbf{v}$, to multiply $\mathbf{v}, \mathbf{M}\mathbf{v}, ..., \mathbf{M}^k\mathbf{v}$ until convergence then it is to compute the eigenvectors of $\mathbf{M}$. In fact, in this case, one needs only compute the first couple of iterates in order to get a good approximation of the PageRank vector. 

\item
For a random matrix, the power method is in general known to be slow to converge. What makes it work fast in this case however is the fact that the web graph is sparse. This means that a node $i$ has a small number of outgoing links (a couple of hundred at best, which is extremely small corresponding to the 30 billion nodes it could theoretically link to). Hence the link matrix $\mathbf{A}$ has a lot of entries equal to 0.


\end{itemize}

\end{document}


































