View source for Wide & Deep Learning for Recommender Systems/paper

<languages />
{{PaperTabs}}
{{PaperInfobox
 | topic_area  = Machine Learning
 | difficulty  = Research
 | authors     = Heng-Tze Cheng; Levent Koc; Jeremiah Harmsen; Tal Shaked; Tushar Chandra; Hrishi Aradhye; Glen Anderson; Greg Corrado; Wei Chai; Mustafa Ispir; Rohan Anil; Zakaria Haque; Lichan Hong; Vihan Jain; Xiaobing Liu; Hemal Shah
 | year        = 2016
 | arxiv_id    = 1606.07792
 | source_url  = https://arxiv.org/abs/1606.07792
 | pdf_url     = https://arxiv.org/pdf/1606.07792.pdf
}}
{{ContentMeta
 | generated_by   = arxiv-importer-raw
 | model_used     = strategy:html5
 | generated_date = 2026-04-27
}}

<translate>
<!--T:1-->
<span id="wide-deep-learning-for-recommender-systems"></span>

<!--T:2-->
= Wide &amp; Deep Learning for Recommender Systems =

<!--T:3-->
Heng-Tze Cheng    Levent Koc    Jeremiah Harmsen    Tal Shaked    Tushar Chandra    Hrishi Aradhye    Glen Anderson    Greg Corrado    Wei Chai    Mustafa Ispir    Rohan Anil    Zakaria Haque    Lichan Hong    Vihan Jain    Xiaobing Liu    Hemal Shah<br />
Google Inc.

<!--T:4-->
<span id="abstract"></span>

<!--T:5-->
== Abstract ==

<!--T:6-->
Generalized linear models with nonlinear feature transformations are widely used for large-scale regression and classification problems with sparse inputs. Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more feature engineering effort. With less feature engineering, deep neural networks can generalize better to unseen feature combinations through low-dimensional dense embeddings learned for the sparse features. However, deep neural networks with embeddings can over-generalize and recommend less relevant items when the user-item interactions are sparse and high-rank. In this paper, we present Wide &amp; Deep learning—jointly trained wide linear models and deep neural networks—to combine the benefits of memorization and generalization for recommender systems. We productionized and evaluated the system on Google Play, a commercial mobile app store with over one billion active users and over one million apps. Online experiment results show that Wide &amp; Deep significantly increased app acquisitions compared with wide-only and deep-only models. We have also open-sourced our implementation in TensorFlow.

<!--T:7-->
<span id="keywords"></span>

<!--T:8-->
== keywords: ==

<!--T:9-->
Wide &amp; Deep Learning, Recommender Systems. <sup>††</sup>titlenote: Corresponding author: <code>hengtze@google.com</code> {CCSXML}

<!--T:10-->
&lt;ccs2012&gt; &lt;concept&gt; &lt;concept_id&gt;10010147.10010257&lt;/concept_id&gt; &lt;concept_desc&gt;Computing methodologies Machine learning&lt;/concept_desc&gt; &lt;concept_significance&gt;500&lt;/concept_significance&gt; &lt;/concept&gt; &lt;concept&gt; &lt;concept_id&gt;10010147.10010257.10010293.10010294&lt;/concept_id&gt; &lt;concept_desc&gt;Computing methodologies Neural networks&lt;/concept_desc&gt; &lt;concept_significance&gt;300&lt;/concept_significance&gt; &lt;/concept&gt; &lt;concept&gt; &lt;concept_id&gt;10010147.10010257.10010258.10010259&lt;/concept_id&gt; &lt;concept_desc&gt;Computing methodologies Supervised learning&lt;/concept_desc&gt; &lt;concept_significance&gt;300&lt;/concept_significance&gt; &lt;/concept&gt; &lt;concept&gt; &lt;concept_id&gt;10002951.10003317.10003347.10003350&lt;/concept_id&gt; &lt;concept_desc&gt;Information systems Recommender systems&lt;/concept_desc&gt; &lt;concept_significance&gt;300&lt;/concept_significance&gt; &lt;/concept&gt; &lt;/ccs2012&gt;

<!--T:11-->
\ccsdesc

<!--T:12-->
[500]Computing methodologies Machine learning \ccsdesc[300]Computing methodologies Neural networks \ccsdesc[300]Computing methodologies Supervised learning \ccsdesc[300]Information systems Recommender systems

<!--T:13-->
\printccsdesc

<!--T:14-->
<div id="S1" class="section">

<!--T:15-->
<span id="introduction"></span>

<!--T:16-->
== 1 Introduction ==

<!--T:17-->
A recommender system can be viewed as a search ranking system, where the input query is a set of user and contextual information, and the output is a ranked list of items. Given a query, the recommendation task is to find the relevant items in a database and then rank the items based on certain objectives, such as clicks or purchases.

<!--T:18-->
One challenge in recommender systems, similar to the general search ranking problem, is to achieve both memorization and generalization. Memorization can be loosely defined as learning the frequent co-occurrence of items or features and exploiting the correlation available in the historical data. Generalization, on the other hand, is based on transitivity of correlation and explores new feature combinations that have never or rarely occurred in the past. Recommendations based on memorization are usually more topical and directly relevant to the items on which users have already performed actions. Compared with memorization, generalization tends to improve the diversity of the recommended items. In this paper, we focus on the apps recommendation problem for the Google Play store, but the approach should apply to generic recommender systems.

<!--T:19-->
For massive-scale online recommendation and ranking systems in an industrial setting, generalized linear models such as logistic regression are widely used because they are simple, scalable and interpretable. The models are often trained on binarized sparse features with one-hot encoding. E.g., the binary feature “<code>user_installed_app=netflix</code>” has value 1 if the user installed Netflix. Memorization can be achieved effectively using cross-product transformations over sparse features, such as <code>AND</code>(<code>user_installed_app=netflix</code>, <code>impression_app=pandora</code>”), whose value is 1 if the user installed Netflix and then is later shown Pandora. This explains how the co-occurrence of a feature pair correlates with the target label. Generalization can be added by using features that are less granular, such as <code>AND</code>(<code>user_installed_category=video</code>, <code>impression_category=music</code>), but manual feature engineering is often required. One limitation of cross-product transformations is that they do not generalize to query-item feature pairs that have not appeared in the training data.

<!--T:20-->
Embedding-based models, such as factorization machines [<nowiki/>[[#bib.bib5|5]]] or deep neural networks, can generalize to previously unseen query-item feature pairs by learning a low-dimensional dense embedding vector for each query and item feature, with less burden of feature engineering. However, it is difficult to learn effective low-dimensional representations for queries and items when the underlying query-item matrix is sparse and high-rank, such as users with specific preferences or niche items with a narrow appeal. In such cases, there should be no interactions between most query-item pairs, but dense embeddings will lead to nonzero predictions for all query-item pairs, and thus can over-generalize and make less relevant recommendations. On the other hand, linear models with cross-product feature transformations can memorize these “exception rules” with much fewer parameters.

<!--T:21-->
<div id="S1.F1" class="figure">

<!--T:22-->
[[File:Arxiv_1606_07792_x1.png|332x60px|Refer to caption]]

<!--T:23-->
In this paper, we present the Wide &amp; Deep learning framework to achieve both memorization and generalization in one model, by jointly training a linear model component and a neural network component as shown in Figure [[#S1.F1|1]].

<!--T:24-->
The main contributions of the paper include:

<!--T:25-->
<ul>
<li><p><span id="S1.I1.i1">•</span></p>
<p>The Wide &amp; Deep learning framework for jointly training feed-forward neural networks with embeddings and linear model with feature transformations for generic recommender systems with sparse inputs.</p></li>
<li><p><span id="S1.I1.i2">•</span></p>
<p>The implementation and evaluation of the Wide &amp; Deep recommender system productionized on Google Play, a mobile app store with over one billion active users and over one million apps.</p></li>
<li><p><span id="S1.I1.i3">•</span></p>
<p>We have open-sourced our implementation along with a high-level API in TensorFlow<sup>11</sup>1See Wide &amp; Deep Tutorial on <code>http://tensorflow.org</code>..</p></li></ul>

<!--T:26-->
While the idea is simple, we show that the Wide &amp; Deep framework significantly improves the app acquisition rate on the mobile app store, while satisfying the training and serving speed requirements.

<!--T:27-->
<div id="S2" class="section">

<!--T:28-->
<span id="recommender-system-overview"></span>

<!--T:29-->
== 2 Recommender System Overview ==

<!--T:30-->
An overview of the app recommender system is shown in Figure [[#S2.F2|2]]. A query, which can include various user and contextual features, is generated when a user visits the app store. The recommender system returns a list of apps (also referred to as impressions) on which users can perform certain actions such as clicks or purchases. These user actions, along with the queries and impressions, are recorded in the logs as the training data for the learner.

<!--T:31-->
Since there are over a million apps in the database, it is intractable to exhaustively score every app for every query within the serving latency requirements (often <math display="inline">O\hspace{0pt}{(10)}</math> milliseconds). Therefore, the first step upon receiving a query is retrieval. The retrieval system returns a short list of items that best match the query using various signals, usually a combination of machine-learned models and human-defined rules. After reducing the candidate pool, the ranking system ranks all items by their scores. The scores are usually <math display="inline">P\hspace{0pt}{(\left. y \middle| \mathbf{x} \right.)}</math>, the probability of a user action label <math display="inline">y</math> given the features <math display="inline">\mathbf{x}</math>, including user features (e.g., country, language, demographics), contextual features (e.g., device, hour of the day, day of the week), and impression features (e.g., app age, historical statistics of an app). In this paper, we focus on the ranking model using the Wide &amp; Deep learning framework.

<!--T:32-->
<div id="S2.F2" class="figure">

<!--T:33-->
[[File:Arxiv_1606_07792_x2.png|332x135px|Refer to caption]]

<!--T:34-->
<div id="S3" class="section">

<!--T:35-->
<span id="wide-deep-learning"></span>

<!--T:36-->
== 3 Wide &amp; Deep Learning ==

<!--T:37-->
<div id="S3.SS1" class="section">

<!--T:38-->
<span id="the-wide-component"></span>

<!--T:39-->
=== 3.1 The Wide Component ===

<!--T:40-->
The wide component is a generalized linear model of the form <math display="inline">y = {{\mathbf{w}^{T}\hspace{0pt}\mathbf{x}} + b}</math>, as illustrated in Figure [[#S1.F1|1]] (left). <math display="inline">y</math> is the prediction, <math display="inline">\mathbf{x} = {\lbrack x_{1},x_{2},\ldots,x_{d}\rbrack}</math> is a vector of <math display="inline">d</math> features, <math display="inline">\mathbf{w} = {\lbrack w_{1},w_{2},\ldots,w_{d}\rbrack}</math> are the model parameters and <math display="inline">b</math> is the bias. The feature set includes raw input features and transformed features. One of the most important transformations is the cross-product transformation, which is defined as:

<!--T:41-->
{| id="S3.E1" class="wikitable"
|-
|
| <math display="block">{{\phi_{k}\hspace{0pt}{(\mathbf{x})}} = {\prod\limits_{i = 1}^{d}x_{i}^{c_{k\hspace{0pt}i}}}}\quad{c_{k\hspace{0pt}i} \in {\{ 0,1\}}}</math>
|
| (1)
|}

<!--T:42-->
where <math display="inline">c_{k\hspace{0pt}i}</math> is a boolean variable that is 1 if the <math display="inline">i</math>-th feature is part of the <math display="inline">k</math>-th transformation <math display="inline">\phi_{k}</math>, and 0 otherwise. For binary features, a cross-product transformation (e.g., “<code>AND</code>(<code>gender=female</code>, <code>language=en</code>)”) is 1 if and only if the constituent features (“<code>gender=female</code>” and “<code>language=en</code>”) are all 1, and 0 otherwise. This captures the interactions between the binary features, and adds nonlinearity to the generalized linear model.

<!--T:43-->
<div id="S3.SS2" class="section">

<!--T:44-->
<span id="the-deep-component"></span>

<!--T:45-->
=== 3.2 The Deep Component ===

<!--T:46-->
The deep component is a feed-forward neural network, as shown in Figure [[#S1.F1|1]] (right). For categorical features, the original inputs are feature strings (e.g., “<code>language=en</code>”). Each of these sparse, high-dimensional categorical features are first converted into a low-dimensional and dense real-valued vector, often referred to as an embedding vector. The dimensionality of the embeddings are usually on the order of <math display="inline">O\hspace{0pt}{(10)}</math> to <math display="inline">O\hspace{0pt}{(100)}</math>. The embedding vectors are initialized randomly and then the values are trained to minimize the final loss function during model training. These low-dimensional dense embedding vectors are then fed into the hidden layers of a neural network in the forward pass. Specifically, each hidden layer performs the following computation:

<!--T:47-->
{| id="S3.E2" class="wikitable"
|-
|
| <math display="block">a^{({l + 1})} = {f\hspace{0pt}{({{W^{(l)}\hspace{0pt}a^{(l)}} + b^{(l)}})}}</math>
|
| (2)
|}

<!--T:48-->
where <math display="inline">l</math> is the layer number and <math display="inline">f</math> is the activation function, often rectified linear units (ReLUs). <math display="inline">a^{(l)}</math>, <math display="inline">b^{(l)}</math>, and <math display="inline">W^{(l)}</math> are the activations, bias, and model weights at <math display="inline">l</math>-th layer.

<!--T:49-->
<div id="S3.SS3" class="section">

<!--T:50-->
<span id="joint-training-of-wide-deep-model"></span>

<!--T:51-->
=== 3.3 Joint Training of Wide &amp; Deep Model ===

<!--T:52-->
The wide component and deep component are combined using a weighted sum of their output log odds as the prediction, which is then fed to one common logistic loss function for joint training. Note that there is a distinction between joint training and ensemble. In an ensemble, individual models are trained separately without knowing each other, and their predictions are combined only at inference time but not at training time. In contrast, joint training optimizes all parameters simultaneously by taking both the wide and deep part as well as the weights of their sum into account at training time. There are implications on model size too: For an ensemble, since the training is disjoint, each individual model size usually needs to be larger (e.g., with more features and transformations) to achieve reasonable accuracy for an ensemble to work. In comparison, for joint training the wide part only needs to complement the weaknesses of the deep part with a small number of cross-product feature transformations, rather than a full-size wide model.

<!--T:53-->
Joint training of a Wide &amp; Deep Model is done by back-propagating the gradients from the output to both the wide and deep part of the model simultaneously using mini-batch stochastic optimization. In the experiments, we used Follow-the-regularized-leader (FTRL) algorithm [<nowiki/>[[#bib.bib3|3]]] with <math display="inline">L_{1}</math> regularization as the optimizer for the wide part of the model, and AdaGrad [<nowiki/>[[#bib.bib1|1]]] for the deep part.

<!--T:54-->
The combined model is illustrated in Figure [[#S1.F1|1]] (center). For a logistic regression problem, the model’s prediction is:

<!--T:55-->
{| id="S3.E3" class="wikitable"
|-
|
| <math display="block">{P\hspace{0pt}{({Y = \left. 1 \middle| \mathbf{x} \right.})}} = {\sigma\hspace{0pt}{({{\mathbf{w}_{w\hspace{0pt}i\hspace{0pt}d\hspace{0pt}e}^{T}\hspace{0pt}{\lbrack\mathbf{x},{\phi\hspace{0pt}{(\mathbf{x})}}\rbrack}} + {\mathbf{w}_{d\hspace{0pt}e\hspace{0pt}e\hspace{0pt}p}^{T}\hspace{0pt}a^{(l_{f})}} + b})}}</math>
|
| (3)
|}

<!--T:56-->
where <math display="inline">Y</math> is the binary class label, <math display="inline">\sigma\hspace{0pt}{( \cdot )}</math> is the sigmoid function, <math display="inline">\phi\hspace{0pt}{(\mathbf{x})}</math> are the cross product transformations of the original features <math display="inline">\mathbf{x}</math>, and <math display="inline">b</math> is the bias term. <math display="inline">\mathbf{w}_{w\hspace{0pt}i\hspace{0pt}d\hspace{0pt}e}</math> is the vector of all wide model weights, and <math display="inline">\mathbf{w}_{d\hspace{0pt}e\hspace{0pt}e\hspace{0pt}p}</math> are the weights applied on the final activations <math display="inline">a^{(l_{f})}</math>.

<!--T:57-->
<div id="S4" class="section">

<!--T:58-->
<span id="system-implementation"></span>

<!--T:59-->
== 4 System Implementation ==

<!--T:60-->
The implementation of the apps recommendation pipeline consists of three stages: data generation, model training, and model serving as shown in Figure [[#S4.F3|3]].

<!--T:61-->
<div id="S4.F3" class="figure">

<!--T:62-->
[[File:Arxiv_1606_07792_x3.png|332x109px|Refer to caption]]

<!--T:63-->
<div id="S4.SS1" class="section">

<!--T:64-->
<span id="data-generation"></span>

<!--T:65-->
=== 4.1 Data Generation ===

<!--T:66-->
In this stage, user and app impression data within a period of time are used to generate training data. Each example corresponds to one impression. The label is app acquisition: 1 if the impressed app was installed, and 0 otherwise.

<!--T:67-->
Vocabularies, which are tables mapping categorical feature strings to integer IDs, are also generated in this stage. The system computes the ID space for all the string features that occurred more than a minimum number of times. Continuous real-valued features are normalized to <math display="inline">\lbrack 0,1\rbrack</math> by mapping a feature value <math display="inline">x</math> to its cumulative distribution function <math display="inline">P\hspace{0pt}{({X \leq x})}</math>, divided into <math display="inline">n_{q}</math> quantiles. The normalized value is <math display="inline">\frac{i - 1}{n_{q} - 1}</math> for values in the <math display="inline">i</math>-th quantiles. Quantile boundaries are computed during data generation.

<!--T:68-->
<div id="S4.SS2" class="section">

<!--T:69-->
<span id="model-training"></span>

<!--T:70-->
=== 4.2 Model Training ===

<!--T:71-->
The model structure we used in the experiment is shown in Figure [[#S4.F4|4]]. During training, our input layer takes in training data and vocabularies and generate sparse and dense features together with a label. The wide component consists of the cross-product transformation of user installed apps and impression apps. For the deep part of the model, A 32-dimensional embedding vector is learned for each categorical feature. We concatenate all the embeddings together with the dense features, resulting in a dense vector of approximately 1200 dimensions. The concatenated vector is then fed into 3 ReLU layers, and finally the logistic output unit.

<!--T:72-->
The Wide &amp; Deep models are trained on over 500 billion examples. Every time a new set of training data arrives, the model needs to be re-trained. However, retraining from scratch every time is computationally expensive and delays the time from data arrival to serving an updated model. To tackle this challenge, we implemented a warm-starting system which initializes a new model with the embeddings and the linear model weights from the previous model.

<!--T:73-->
<div id="S4.F4" class="figure">

<!--T:74-->
[[File:Arxiv_1606_07792_x4.png|332x133px|Refer to caption]]

<!--T:75-->
Before loading the models into the model servers, a dry run of the model is done to make sure that it does not cause problems in serving live traffic. We empirically validate the model quality against the previous model as a sanity check.

<!--T:76-->
<div id="S4.SS3" class="section">

<!--T:77-->
<span id="model-serving"></span>

<!--T:78-->
=== 4.3 Model Serving ===

<!--T:79-->
Once the model is trained and verified, we load it into the model servers. For each request, the servers receive a set of app candidates from the app retrieval system and user features to score each app. Then, the apps are ranked from the highest scores to the lowest, and we show the apps to the users in this order. The scores are calculated by running a forward inference pass over the Wide &amp; Deep model.

<!--T:80-->
In order to serve each request on the order of 10 ms, we optimized the performance using multithreading parallelism by running smaller batches in parallel, instead of scoring all candidate apps in a single batch inference step.

<!--T:81-->
<div id="S5" class="section">

<!--T:82-->
<span id="experiment-results"></span>

<!--T:83-->
== 5 Experiment Results ==

<!--T:84-->
To evaluate the effectiveness of Wide &amp; Deep learning in a real-world recommender system, we ran live experiments and evaluated the system in a couple of aspects: app acquisitions and serving performance.

<!--T:85-->
<div id="S5.T1" class="figure">

<!--T:86-->
{| id="S5.T1.1" class="wikitable"
|- id="S5.T1.1.1.1"
! id="S5.T1.1.1.1.1"| Model
! id="S5.T1.1.1.1.2"| Offline AUC
! id="S5.T1.1.1.1.3"| Online Acquisition Gain
|- id="S5.T1.1.2.1"
! id="S5.T1.1.2.1.1"| Wide (control)
| id="S5.T1.1.2.1.2"| 0.726
| id="S5.T1.1.2.1.3"| 0%
|- id="S5.T1.1.3.2"
! id="S5.T1.1.3.2.1"| Deep
| id="S5.T1.1.3.2.2"| 0.722
| id="S5.T1.1.3.2.3"| +2.9%
|- id="S5.T1.1.4.3"
! id="S5.T1.1.4.3.1"| Wide &amp; Deep
| id="S5.T1.1.4.3.2"| 0.728
| id="S5.T1.1.4.3.3"| +3.9%
|}

<!--T:87-->
<div id="S5.SS1" class="section">

<!--T:88-->
<span id="app-acquisitions"></span>

<!--T:89-->
=== 5.1 App Acquisitions ===

<!--T:90-->
We conducted live online experiments in an A/B testing framework for 3 weeks. For the control group, 1% of users were randomly selected and presented with recommendations generated by the previous version of ranking model, which is a highly-optimized wide-only logistic regression model with rich cross-product feature transformations. For the experiment group, 1% of users were presented with recommendations generated by the Wide &amp; Deep model, trained with the same set of features. As shown in Table [[#S5.T1|1]], Wide &amp; Deep model improved the app acquisition rate on the main landing page of the app store by +3.9% relative to the control group (statistically significant). The results were also compared with another 1% group using only the deep part of the model with the same features and neural network structure, and the Wide &amp; Deep mode had +1% gain on top of the deep-only model (statistically significant).

<!--T:91-->
Besides online experiments, we also show the Area Under Receiver Operator Characteristic Curve (AUC) on a holdout set offline. While Wide &amp; Deep has a slightly higher offline AUC, the impact is more significant on online traffic. One possible reason is that the impressions and labels in offline data sets are fixed, whereas the online system can generate new exploratory recommendations by blending generalization with memorization, and learn from new user responses.

<!--T:92-->
<div id="S5.SS2" class="section">

<!--T:93-->
<span id="serving-performance"></span>

<!--T:94-->
=== 5.2 Serving Performance ===

<!--T:95-->
Serving with high throughput and low latency is challenging with the high level of traffic faced by our commercial mobile app store. At peak traffic, our recommender servers score over 10 million apps per second. With single threading, scoring all candidates in a single batch takes 31 ms. We implemented multithreading and split each batch into smaller sizes, which significantly reduced the client-side latency to 14 ms (including serving overhead) as shown in Table [[#S7.T2|2]].

<!--T:96-->
<div id="S6" class="section">

<!--T:97-->
<span id="related-work"></span>

<!--T:98-->
== 6 Related Work ==

<!--T:99-->
The idea of combining wide linear models with cross-product feature transformations and deep neural networks with dense embeddings is inspired by previous work, such as factorization machines [<nowiki/>[[#bib.bib5|5]]] which add generalization to linear models by factorizing the interactions between two variables as a dot product between two low-dimensional embedding vectors. In this paper, we expanded the model capacity by learning highly nonlinear interactions between embeddings via neural networks instead of dot products.

<!--T:100-->
In language models, joint training of recurrent neural networks (RNNs) and maximum entropy models with n-gram features has been proposed to significantly reduce the RNN complexity (e.g., hidden layer sizes) by learning direct weights between inputs and outputs [<nowiki/>[[#bib.bib4|4]]]. In computer vision, deep residual learning [<nowiki/>[[#bib.bib2|2]]] has been used to reduce the difficulty of training deeper models and improve accuracy with shortcut connections which skip one or more layers. Joint training of neural networks with graphical models has also been applied to human pose estimation from images [<nowiki/>[[#bib.bib6|6]]]. In this work we explored the joint training of feed-forward neural networks and linear models, with direct connections between sparse features and the output unit, for generic recommendation and ranking problems with sparse input data.

<!--T:101-->
In the recommender systems literature, collaborative deep learning has been explored by coupling deep learning for content information and collaborative filtering (CF) for the ratings matrix [<nowiki/>[[#bib.bib7|7]]]. There has also been previous work on mobile app recommender systems, such as AppJoy which used CF on users’ app usage records [<nowiki/>[[#bib.bib8|8]]]. Different from the CF-based or content-based approaches in the previous work, we jointly train Wide &amp; Deep models on user and impression data for app recommender systems.

<!--T:102-->
<div id="S7" class="section">

<!--T:103-->
<span id="conclusion"></span>

<!--T:104-->
== 7 Conclusion ==

<!--T:105-->
Memorization and generalization are both important for recommender systems. Wide linear models can effectively memorize sparse feature interactions using cross-product feature transformations, while deep neural networks can generalize to previously unseen feature interactions through low-dimensional embeddings. We presented the Wide &amp; Deep learning framework to combine the strengths of both types of model. We productionized and evaluated the framework on the recommender system of Google Play, a massive-scale commercial app store. Online experiment results showed that the Wide &amp; Deep model led to significant improvement on app acquisitions over wide-only and deep-only models.

<!--T:106-->
<div id="S7.T2" class="figure">

<!--T:107-->
{| id="S7.T2.1" class="wikitable"
|- id="S7.T2.1.1.1"
| id="S7.T2.1.1.1.1"| Batch size
| id="S7.T2.1.1.1.2"| Number of Threads
| id="S7.T2.1.1.1.3"| Serving Latency (ms)
|- id="S7.T2.1.2.2"
| id="S7.T2.1.2.2.1"| 200
| id="S7.T2.1.2.2.2"| 1
| id="S7.T2.1.2.2.3"| 31
|- id="S7.T2.1.3.3"
| id="S7.T2.1.3.3.1"| 100
| id="S7.T2.1.3.3.2"| 2
| id="S7.T2.1.3.3.3"| 17
|- id="S7.T2.1.4.4"
| id="S7.T2.1.4.4.1"| 50
| id="S7.T2.1.4.4.2"| 4
| id="S7.T2.1.4.4.3"| 14
|}

<!--T:108-->
<div id="bib" class="section">

<!--T:109-->
<span id="references"></span>

<!--T:110-->
== References ==

<!--T:111-->
* <span id="bib.bib1">[1] J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12:2121–2159, July 2011.</span>
* <span id="bib.bib2">[2] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2016.</span>
* <span id="bib.bib3">[3] H. B. McMahan. Follow-the-regularized-leader and mirror descent: Equivalence theorems and l1 regularization. In Proc. AISTATS, 2011.</span>
* <span id="bib.bib4">[4] T. Mikolov, A. Deoras, D. Povey, L. Burget, and J. H. Cernocky. Strategies for training large scale neural network language models. In IEEE Automatic Speech Recognition &amp; Understanding Workshop, 2011.</span>
* <span id="bib.bib5">[5] S. Rendle. Factorization machines with libFM. ACM Trans. Intell. Syst. Technol., 3(3):57:1–57:22, May 2012.</span>
* <span id="bib.bib6">[6] J. J. Tompson, A. Jain, Y. LeCun, and C. Bregler. Joint training of a convolutional network and a graphical model for human pose estimation. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, NIPS, pages 1799–1807. 2014.</span>
* <span id="bib.bib7">[7] H. Wang, N. Wang, and D.-Y. Yeung. Collaborative deep learning for recommender systems. In Proc. KDD, pages 1235–1244, 2015.</span>
* <span id="bib.bib8">[8] B. Yan and G. Chen. AppJoy: Personalized mobile application discovery. In MobiSys, pages 113–126, 2011.</span>
</translate>

[[Category:Machine Learning]]
[[Category:Research]]
[[Category:Research Papers]]