rtbgym.envs.simulator.rtb_synthetic.RTBSyntheticSimulator#

class rtbgym.envs.simulator.rtb_synthetic.RTBSyntheticSimulator(cost_indicator='click', step_per_episode=7, n_ads=100, n_users=100, ad_feature_dim=5, user_feature_dim=5, ad_feature_vector=None, user_feature_vector=None, ad_sampling_rate=None, user_sampling_rate=None, WinningPriceDistribution=<class 'rtbgym.envs.simulator.function.WinningPriceDistribution'>, ClickThroughRate=<class 'rtbgym.envs.simulator.function.ClickThroughRate'>, ConversionRate=<class 'rtbgym.envs.simulator.function.ConversionRate'>, standard_bid_price_distribution=(None, ), minimum_standard_bid_price=None, search_volume_distribution=(None, ), minimum_search_volume=10, random_state=None)[source]#

Class to calculate the outcome probability and stochastically determine auction result in Real-Time Bidding (RTB) setting for display advertising.

Imported as: rtbgym.envs.simulator.RTBSyntheticSimulator

Parameters:

  • cost_indicator ({"impression", "click", "conversion"}, default="click") – Defines when the cost arises.

  • step_per_episode (int, default=7 (> 0)) – Number of timesteps in an episode.

  • n_ads (int, default=100 (> 0)) – Number of (candidate) ads used for auction bidding.

  • n_users (int, default=100 (> 0)) – Number of (candidate) users used for auction bidding.

  • ad_feature_vector (array-like of shape (n_ads, ad_feature_dim), default=None) – Feature vectors that characterizes each ad.

  • user_feature_vector (array-like of shape (n_users, user_feature_dim), default=None) – Feature vectors that characterizes each user.

  • ad_sampling_rate (array-like of shape (step_per_episode, n_ads) or (n_ads, ), default=None) – Sampling probabilities to determine which ad (id) is used in each auction.

  • user_sampling_rate (array-like of shape (step_per_episode, n_users) or (n_uses, ), default=None) – Sampling probabilities to determine which user (id) is used in each auction.

  • WinningPriceDistribution (BaseWinningPriceDistribution) – Winning price distribution of auctions. Both class and instance are acceptable.

  • ClickThroughRate (BaseClickAndConversionRate) – Click through rate (i.e., click / impression). Both class and instance are acceptable.

  • ConversionRate (BaseClickAndConversionRate) – Conversion rate (i.e., conversion / click). Both class and instance are acceptable.

  • standard_bid_price_distribution (NormalDistribution, default=None) – Distribution of the bid price whose average impression probability is expected to be 0.5.

  • minimum_standard_bid_price (int, default=None (> 0)) – Minimum value for standard bid price. If None, minimum_standard_bid_price is set to standard_bid_price_distribution.mean / 2.

  • search_volume_distribution (NormalDistribution, default=None) – Search volume distribution for each timestep.

  • minimum_search_volume (int, default = 10 (> 0)) – Minimum search volume at each timestep.

  • random_state (int, default=None (>= 0)) – Random state.

References

Di Wu, Xiujun Chen, Xun Yang, Hao Wang, Qing Tan, Xiaoxun Zhang, Jian Xu, and Kun Gai. “Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising.” 2018.

Jun Zhao, Guang Qiu, Ziyu Guan, Wei Zhao, and Xiaofei He. “Deep Reinforcement Learning for Sponsored Search Real-time Bidding.” 2018.

Attributes:
ad_feature_vector
ad_sampling_rate
minimum_standard_bid_price
random_state
standard_bid_price
user_feature_vector
user_sampling_rate

Methods

ClickThroughRate(n_ads, n_users, ...[, ...])

Class to calculate ground-truth CTR (i.e., click per impression).

ConversionRate(n_ads, n_users, ...[, ...])

Class to calculate ground-truth CVR (i.e., conversion per click).

WinningPriceDistribution(n_ads, n_users, ...)

Class to sample the winning price (i.e., second price) and compare it with the given bid price.

calc_and_sample_outcome(timestep, ad_ids, ...)

Simulate bidding auction for given queries.

generate_auction([volume, timestep])

Sample ad and user pair for each auction.

map_idx_to_features(ad_ids, user_ids)

Map the ad and the user index into feature vectors.
class WinningPriceDistribution(n_ads, n_users, ad_feature_dim, user_feature_dim, step_per_episode, standard_bid_price_distribution=(None,), minimum_standard_bid_price=None, random_state=None)#

Class to sample the winning price (i.e., second price) and compare it with the given bid price.

Imported as: rtbgym.envs.simulator.WinningDistribution

Note

Winning price distribution follows gamma distribution.

\[p(x) = x^{k-1} \frac{\exp(- x / \theta)}{\theta^k \Gamma(k)},\]

where \(\Gamma(k) := (k-1)!\) and \(k\) and \(\theta\) are hyperparameters.

Tip

Use BaseWinningPriceDistribution to define a custom WinningPriceDistribution.

Parameters:

  • n_ads (int (> 0)) – Number of ads.

  • n_users (int (> 0)) – Number of users. (This is for API consistency)

  • ad_feature_dim (int (> 0)) – Dimension of the ad feature vectors. (This is for API consistency)

  • user_feature_dim (int (> 0)) – Dimension of the user feature vectors. (This is for API consistency)

  • step_per_episode (int (> 0)) – Length of the CTR trend cycle. (This is for API consistency)

  • standard_bid_price_distribution (NormalDistribution, default=None) – Distribution of the bid price whose average impression probability is expected to be 0.5.

  • minimum_standard_bid_price ({int, float}, default=None (> 0)) – Minimum value for standard bid price. If None, minimum_standard_bid_price is set to standard_bid_price_distribution.mean / 2.

  • random_state (int, default=None (>= 0)) – Random state.

References

Wen-Yuan Zhu, Wen-Yueh Shih, Ying-Hsuan Lee, Wen-Chih Peng, and Jiun-Long Huang. “A Gamma-based Regression for Winning Price Estimation in Real-Time Bidding Advertising.” 2017.

Attributes:
minimum_standard_bid_price
random_state
standard_bid_price

Methods

sample_outcome(bid_prices, ad_ids, user_ids, ...)

Calculate impression probability for given bid price.
sample_outcome(bid_prices, ad_ids, user_ids, ad_feature_vector, user_feature_vector, timestep)#

Calculate impression probability for given bid price.

Parameters:

  • bid_prices (array-like of shape (search_volume, )) – Bid price for each auction.

  • ad_ids (array-like of shape (search_volume/n_samples, )) – Ad ids used for each auction. (This is for API consistency)

  • user_ids (array-like of shape (search_volume/n_samples, )) – User ids used for each auction. (This is for API consistency)

  • ad_feature_vector (array-like of shape (search_volume/n_samples, ad_feature_dim)) – Ad feature vector for each auction.

  • user_feature_vector (array-like of shape (search_volume/n_samples, user_feature_dim)) – User feature vector for each auction.

  • timestep ({int, array-like of shape (n_samples, )}) – Timestep in the RL environment.

Returns:

  • impressions (ndarray of shape (search_volume, )) – Whether impression occurred for each auction.

  • winning_prices (ndarray of shape (search_volume, )) – Sampled winning price for each auction.

Return type:

Tuple[ndarray]

class ClickThroughRate(n_ads, n_users, ad_feature_dim, user_feature_dim, step_per_episode, random_state=None)#

Class to calculate ground-truth CTR (i.e., click per impression).

Imported as: rtbgym.envs.simulator.ClickThroughRate

Note

We define two coefficient, context coefficient (coef) and time coefficient (time_coef). First, the value is calculated linearly from context vector and coef by inner product. Then, we multiply the value with time_coef and gain (ground-truth) CTR.

In short, CTR is calculated as follows.

CTR = (context @ coef) * time_coef, where @ denotes inner product.

Tip

Use BaseClickAndConversionRate to define a custom ClickThroughRate.

Parameters:

  • n_ads (int (> 0)) – Number of ads. (This is for API consistency)

  • n_users (int (> 0)) – Number of users. (This is for API consistency)

  • ad_feature_dim (int (> 0)) – Dimension of the ad feature vectors.

  • user_feature_dim (int (> 0)) – Dimension of the user feature vectors.

  • step_per_episode (int (> 0)) – Length of the CTR trend cycle.

  • random_state (int, default=None (>= 0)) – Random state.

Attributes:
random_state

Methods

calc_prob(ad_ids, user_ids, ...)

Calculate CTR (i.e., click per impression).

sample_outcome(ad_ids, user_ids, ...)

Stochastically determine whether click occurs in impression=True case.
calc_prob(ad_ids, user_ids, ad_feature_vector, user_feature_vector, timestep)#

Calculate CTR (i.e., click per impression).

Note

CTR is calculated using both context coefficient (coef) and time coefficient (time_coef).

CTR = (context @ coef) * time_coef, where @ denotes inner product.

Parameters:

  • ad_ids (array-like of shape (search_volume/n_samples, )) – Ad ids used for each auction. (not used, but for API consistency)

  • user_ids (array-like of shape (search_volume/n_samples, )) – User ids used for each auction. (not used, but for API consistency)

  • ad_feature_vector (array-like of shape (search_volume/n_samples, ad_feature_dim)) – Ad feature vector for each auction.

  • user_feature_vector (array-like of shape (search_volume/n_samples, user_feature_dim)) – User feature vector for each auction.

  • timestep ({int, array-like of shape (n_samples, )}) – Timestep in the RL environment.

Returns:

ctrs – Ground-truth CTR (i.e., click per impression) for each auction.

Return type:

ndarray of shape (search_volume/n_samples, )

sample_outcome(ad_ids, user_ids, ad_feature_vector, user_feature_vector, timestep)#

Stochastically determine whether click occurs in impression=True case.

Parameters:

  • ad_ids (array-like of shape (search_volume/n_samples, )) – Ad ids used for each auction. (not used, but for API consistency)

  • user_ids (array-like of shape (search_volume/n_samples, )) – User ids used for each auction. (not used, but for API consistency)

  • ad_feature_vector (array-like of shape (search_volume/n_samples, ad_feature_dim)) – Ad feature vector for each auction.

  • user_feature_vector (array-like of shape (search_volume/n_samples, user_feature_dim)) – User feature vector for each auction.

  • timestep ({int, array-like of shape (n_samples, )}) – Timestep in the RL environment.

Returns:

clicks – Whether click occurs when impression=True.

Return type:

array-like of shape (search_volume/n_samples, )

class ConversionRate(n_ads, n_users, ad_feature_dim, user_feature_dim, step_per_episode, random_state=None)#

Class to calculate ground-truth CVR (i.e., conversion per click).

Imported as: rtbgym.envs.simulator.ConversionRate

Note

We define two coefficient, context coefficient (coef) and time coefficient (time_coef). First, the value is calculated linearly from context vector and coef by inner product. Then, we multiply the value with time_coef and gain (ground-truth) CVR.

In short, CVR is calculated as follows.

CVR = (context @ coef) * time_coef, where @ denotes inner product.

Tip

Use BaseClickAndConversionRate to define a custom ConversionRate.

Parameters:

  • n_ads (int (> 0)) – Number of ads. (This is for API consistency)

  • n_users (int (> 0)) – Number of users. (This is for API consistency)

  • ad_feature_dim (int (> 0)) – Dimension of the ad feature vectors.

  • user_feature_dim (int (> 0)) – Dimension of the user feature vectors.

  • step_per_episode (int (> 0)) – Length of the CVR trend cycle.

  • random_state (int, default=None (>= 0)) – Random state.

Attributes:
random_state

Methods

calc_prob(ad_ids, user_ids, ...)

Calculate CVR (i.e., conversion per click) using context vectors.

sample_outcome(ad_ids, user_ids, ...)

Stochastically determine whether conversion occurs in click=True case.
calc_prob(ad_ids, user_ids, ad_feature_vector, user_feature_vector, timestep)#

Calculate CVR (i.e., conversion per click) using context vectors.

Note

CVR is calculated using both context coefficient (coef) and time coefficient (time_coef).

CVR = (context @ coef) * time_coef, where @ denotes inner product.

Parameters:

  • ad_ids (array-like of shape (search_volume/n_samples, )) – Ad ids used for each auction. (not used, but for API consistency)

  • user_ids (array-like of shape (search_volume/n_samples, )) – User ids used for each auction. (not used, but for API consistency)

  • ad_feature_vector (array-like of shape (search_volume/n_samples, ad_feature_dim)) – Ad feature vector for each auction.

  • user_feature_vector (array-like of shape (search_volume/n_samples, user_feature_dim)) – User feature vector for each auction.

  • timestep ({int, array-like of shape (n_samples, )}) – Timestep in the RL environment.

Returns:

cvrs – Ground-truth CVR (i.e., conversion per click) for each auction.

Return type:

ndarray of shape (search_volume/n_samples, )

sample_outcome(ad_ids, user_ids, ad_feature_vector, user_feature_vector, timestep)#

Stochastically determine whether conversion occurs in click=True case.

Parameters:

  • ad_ids (array-like of shape (search_volume/n_samples, )) – Ad ids used for each auction. (not used, but for API consistency)

  • user_ids (array-like of shape (search_volume/n_samples, )) – User ids used for each auction. (not used, but for API consistency)

  • ad_feature_vector (array-like of shape (search_volume/n_samples, ad_feature_dim)) – Ad feature vector for each auction.

  • user_feature_vector (array-like of shape (search_volume/n_samples, user_feature_dim)) – User feature vector for each auction.

  • timestep ({int, array-like of shape (n_samples, )}) – Timestep in the RL environment.

Returns:

conversions – Whether conversion occurs when click=True.

Return type:

ndarray of shape (search_volume/n_samples, )

generate_auction(volume=None, timestep=None)[source]#

Sample ad and user pair for each auction.

Parameters:

  • volume (int, default=None (> 0)) – Total number of auctions to generate.

  • timestep (int, default=None (> 0)) – Timestep in the RL environment.

Returns:

  • ad_ids (ndarray of shape (volume, )) – IDs of the ads.

  • user_ids (ndarray of shape (volume, )) – IDs of the users.

map_idx_to_features(ad_ids, user_ids)[source]#

Map the ad and the user index into feature vectors.

Parameters:

  • ad_ids (array-like of shape (search_volume, )) – IDs of the ads. (search_volume is determined in RL environment.)

  • user_ids (array-like of shape (search_volume, )) – IDs of the users. (search_volume is determined in RL environment.)

Returns:

  • ad_feature_vector (ndarray of shape (search_volume/n_samples, ad_feature_dim)) – Ad feature vector for each auction.

  • user_feature_vector (ndarray of shape (search_volume/n_samples, user_feature_dim)) – User feature vector for each auction.

Return type:

ndarray

calc_and_sample_outcome(timestep, ad_ids, user_ids, bid_prices)[source]#

Simulate bidding auction for given queries. (Calculate outcome probability and stochastically determine auction result.)

Parameters:

  • timestep (int (> 0)) – Timestep in the RL environment.

  • ad_ids (array-like of shape (search_volume, )) – IDs of the ads.

  • user_ids (array-like of shape (search_volume, )) – IDs of the users.

  • bid_prices (array-like of shape(search_volume, )) – Bid price for each action. (search_volume is determined in RL environment.)

Returns:

  • costs (ndarray of shape (search_volume, )) – Cost raised (i.e., second price) for each auction.

  • impressions (ndarray of shape (search_volume, )) – Binary indicator of whether impression occurred or not for each auction.

  • clicks (ndarray of shape (search_volume, )) – Binary indicator of whether click occurred or not for each auction.

  • conversions (ndarray of shape (search_volume, )) – Binary indicator of whether conversion occurred or not for each auction.

Return type:

Tuple[ndarray]

Methods