Skip to main content
Ctrl+K
Logo image

Site Navigation

  • Installation
  • Quickstart
  • Documentation
  • Usage
  • API
  • Sub-packages
  • FAQs
  • News
  • Release Notes
  • Proceedings

Site Navigation

  • Installation
  • Quickstart
  • Documentation
  • Usage
  • API
  • Sub-packages
  • FAQs
  • News
  • Release Notes
  • Proceedings

Section Navigation

Getting Started:

  • Installation
  • Quickstart
  • Why SCOPE-RL?

Online & Offline RL:

  • Overview
  • Supported Implementation

Off-Policy Evaluation & Selection:

  • Overview
  • Supported Implementation
  • Visualization Tools

Our Proposal:

  • Risk-Return Assessments of OPE via SharpeRatio@k

Sub-packages:

  • Gallery of Sub-packages

Package References:

  • SCOPE-RL Package Reference
    • scope_rl.dataset.base
      • scope_rl.dataset.base.BaseDataset
    • scope_rl.dataset.synthetic
      • scope_rl.dataset.synthetic.SyntheticDataset
    • scope_rl.policy.head
      • scope_rl.policy.head.BaseHead
      • scope_rl.policy.head.ContinuousEvalHead
      • scope_rl.policy.head.EpsilonGreedyHead
      • scope_rl.policy.head.GaussianHead
      • scope_rl.policy.head.OnlineHead
      • scope_rl.policy.head.SoftmaxHead
      • scope_rl.policy.head.TruncatedGaussianHead
    • scope_rl.policy.orl
      • scope_rl.policy.orl.TrainCandidatePolicies
    • scope_rl.ope.input
      • scope_rl.ope.input.CreateOPEInput
    • scope_rl.ope.ope
      • scope_rl.ope.ope.CumulativeDistributionOPE
      • scope_rl.ope.ope.OffPolicyEvaluation
    • scope_rl.ope.ops
      • scope_rl.ope.ops.OffPolicySelection
    • scope_rl.ope.estimators_base
      • scope_rl.ope.estimators_base.BaseCumulativeDistributionOPEEstimator
      • scope_rl.ope.estimators_base.BaseMarginalOPEEstimator
      • scope_rl.ope.estimators_base.BaseOffPolicyEstimator
      • scope_rl.ope.estimators_base.BaseStateActionMarginalOPEEstimator
      • scope_rl.ope.estimators_base.BaseStateMarginalOPEEstimator
    • scope_rl.ope.discrete.basic_estimators
      • scope_rl.ope.discrete.basic_estimators.DirectMethod
      • scope_rl.ope.discrete.basic_estimators.DoublyRobust
      • scope_rl.ope.discrete.basic_estimators.PerDecisionImportanceSampling
      • scope_rl.ope.discrete.basic_estimators.SelfNormalizedDR
      • scope_rl.ope.discrete.basic_estimators.SelfNormalizedPDIS
      • scope_rl.ope.discrete.basic_estimators.SelfNormalizedTIS
      • scope_rl.ope.discrete.basic_estimators.TrajectoryWiseImportanceSampling
    • scope_rl.ope.continuous.basic_estimators
      • scope_rl.ope.continuous.basic_estimators.DirectMethod
      • scope_rl.ope.continuous.basic_estimators.DoublyRobust
      • scope_rl.ope.continuous.basic_estimators.PerDecisionImportanceSampling
      • scope_rl.ope.continuous.basic_estimators.SelfNormalizedDR
      • scope_rl.ope.continuous.basic_estimators.SelfNormalizedPDIS
      • scope_rl.ope.continuous.basic_estimators.SelfNormalizedTIS
      • scope_rl.ope.continuous.basic_estimators.TrajectoryWiseImportanceSampling
    • scope_rl.ope.discrete.marginal_estimators
      • scope_rl.ope.discrete.marginal_estimators.DoubleReinforcementLearning
      • scope_rl.ope.discrete.marginal_estimators.StateActionMarginalDR
      • scope_rl.ope.discrete.marginal_estimators.StateActionMarginalIS
      • scope_rl.ope.discrete.marginal_estimators.StateActionMarginalSNDR
      • scope_rl.ope.discrete.marginal_estimators.StateActionMarginalSNIS
      • scope_rl.ope.discrete.marginal_estimators.StateMarginalDM
      • scope_rl.ope.discrete.marginal_estimators.StateMarginalDR
      • scope_rl.ope.discrete.marginal_estimators.StateMarginalIS
      • scope_rl.ope.discrete.marginal_estimators.StateMarginalSNDR
      • scope_rl.ope.discrete.marginal_estimators.StateMarginalSNIS
    • scope_rl.ope.continuous.marginal_estimators
      • scope_rl.ope.continuous.marginal_estimators.DoubleReinforcementLearning
      • scope_rl.ope.continuous.marginal_estimators.StateActionMarginalDR
      • scope_rl.ope.continuous.marginal_estimators.StateActionMarginalIS
      • scope_rl.ope.continuous.marginal_estimators.StateActionMarginalSNDR
      • scope_rl.ope.continuous.marginal_estimators.StateActionMarginalSNIS
      • scope_rl.ope.continuous.marginal_estimators.StateMarginalDM
      • scope_rl.ope.continuous.marginal_estimators.StateMarginalDR
      • scope_rl.ope.continuous.marginal_estimators.StateMarginalIS
      • scope_rl.ope.continuous.marginal_estimators.StateMarginalSNDR
      • scope_rl.ope.continuous.marginal_estimators.StateMarginalSNIS
    • scope_rl.ope.discrete.cumulative_distribution_estimators
      • scope_rl.ope.discrete.cumulative_distribution_estimators.CumulativeDistributionDM
      • scope_rl.ope.discrete.cumulative_distribution_estimators.CumulativeDistributionSNTDR
      • scope_rl.ope.discrete.cumulative_distribution_estimators.CumulativeDistributionSNTIS
      • scope_rl.ope.discrete.cumulative_distribution_estimators.CumulativeDistributionTDR
      • scope_rl.ope.discrete.cumulative_distribution_estimators.CumulativeDistributionTIS
    • scope_rl.ope.continuous.cumulative_distribution_estimators
      • scope_rl.ope.continuous.cumulative_distribution_estimators.CumulativeDistributionDM
      • scope_rl.ope.continuous.cumulative_distribution_estimators.CumulativeDistributionSNTDR
      • scope_rl.ope.continuous.cumulative_distribution_estimators.CumulativeDistributionSNTIS
      • scope_rl.ope.continuous.cumulative_distribution_estimators.CumulativeDistributionTDR
      • scope_rl.ope.continuous.cumulative_distribution_estimators.CumulativeDistributionTIS
    • scope_rl.ope.weight_value_learning.base
      • scope_rl.ope.weight_value_learning.base.BaseWeightValueLearner
    • scope_rl.ope.weight_value_learning.function
      • scope_rl.ope.weight_value_learning.function.ContinuousQFunction
      • scope_rl.ope.weight_value_learning.function.ContinuousStateActionWeightFunction
      • scope_rl.ope.weight_value_learning.function.DiscreteQFunction
      • scope_rl.ope.weight_value_learning.function.DiscreteStateActionWeightFunction
      • scope_rl.ope.weight_value_learning.function.StateWeightFunction
      • scope_rl.ope.weight_value_learning.function.VFunction
    • scope_rl.ope.weight_value_learning.augmented_lagrangian_learning_discrete
      • scope_rl.ope.weight_value_learning.augmented_lagrangian_learning_discrete.DiscreteDiceStateActionWightValueLearning
      • scope_rl.ope.weight_value_learning.augmented_lagrangian_learning_discrete.DiscreteDiceStateWightValueLearning
    • scope_rl.ope.weight_value_learning.augmented_lagrangian_learning_continuous
      • scope_rl.ope.weight_value_learning.augmented_lagrangian_learning_continuous.ContinuousDiceStateActionWightValueLearning
      • scope_rl.ope.weight_value_learning.augmented_lagrangian_learning_continuous.ContinuousDiceStateWightValueLearning
    • scope_rl.ope.weight_value_learning.minimax_weight_learning_discrete
      • scope_rl.ope.weight_value_learning.minimax_weight_learning_discrete.DiscreteMinimaxStateActionWeightLearning
      • scope_rl.ope.weight_value_learning.minimax_weight_learning_discrete.DiscreteMinimaxStateWeightLearning
    • scope_rl.ope.weight_value_learning.minimax_weight_learning_continuous
      • scope_rl.ope.weight_value_learning.minimax_weight_learning_continuous.ContinuousMinimaxStateActionWeightLearning
      • scope_rl.ope.weight_value_learning.minimax_weight_learning_continuous.ContinuousMinimaxStateWeightLearning
    • scope_rl.ope.weight_value_learning.minimax_value_learning_discrete
      • scope_rl.ope.weight_value_learning.minimax_value_learning_discrete.DiscreteMinimaxStateActionValueLearning
      • scope_rl.ope.weight_value_learning.minimax_value_learning_discrete.DiscreteMinimaxStateValueLearning
    • scope_rl.ope.weight_value_learning.minimax_value_learning_continuous
      • scope_rl.ope.weight_value_learning.minimax_value_learning_continuous.ContinuousMinimaxStateActionValueLearning
      • scope_rl.ope.weight_value_learning.minimax_value_learning_continuous.ContinuousMinimaxStateValueLearning
    • scope_rl.ope.online
      • scope_rl.ope.online.calc_on_policy_conditional_value_at_risk
      • scope_rl.ope.online.calc_on_policy_cumulative_distribution_function
      • scope_rl.ope.online.calc_on_policy_interquartile_range
      • scope_rl.ope.online.calc_on_policy_policy_value
      • scope_rl.ope.online.calc_on_policy_policy_value_interval
      • scope_rl.ope.online.calc_on_policy_statistics
      • scope_rl.ope.online.calc_on_policy_variance
      • scope_rl.ope.online.rollout_policy_online
      • scope_rl.ope.online.visualize_on_policy_conditional_value_at_risk
      • scope_rl.ope.online.visualize_on_policy_cumulative_distribution_function
      • scope_rl.ope.online.visualize_on_policy_interquartile_range
      • scope_rl.ope.online.visualize_on_policy_policy_value
      • scope_rl.ope.online.visualize_on_policy_policy_value_with_variance
    • scope_rl.utils
      • scope_rl.utils.check_array
      • scope_rl.utils.check_input_dict
      • scope_rl.utils.check_logged_dataset
      • scope_rl.utils.cosine_kernel
      • scope_rl.utils.defaultdict_to_dict
      • scope_rl.utils.epanechnikov_kernel
      • scope_rl.utils.estimate_confidence_interval_by_bootstrap
      • scope_rl.utils.estimate_confidence_interval_by_empirical_bernstein
      • scope_rl.utils.estimate_confidence_interval_by_hoeffding
      • scope_rl.utils.estimate_confidence_interval_by_t_test
      • scope_rl.utils.gaussian_kernel
      • scope_rl.utils.l2_distance
      • scope_rl.utils.triangular_kernel
      • scope_rl.utils.uniform_kernel
      • scope_rl.utils.MultipleInputDict
      • scope_rl.utils.MultipleLoggedDataset
      • scope_rl.utils.NewGymAPIWrapper
      • scope_rl.utils.OldGymAPIWrapper
  • RTBGym Package Reference
    • rtbgym.envs.rtb
      • rtbgym.envs.rtb.RTBEnv
    • rtbgym.envs.wrapper_rtb
      • rtbgym.envs.wrapper_rtb.CustomizedRTBEnv
    • rtbgym.envs.simulator.base
      • rtbgym.envs.simulator.base.BaseClickAndConversionRate
      • rtbgym.envs.simulator.base.BaseSimulator
      • rtbgym.envs.simulator.base.BaseWinningPriceDistribution
    • rtbgym.envs.simulator.function
      • rtbgym.envs.simulator.function.ClickThroughRate
      • rtbgym.envs.simulator.function.ConversionRate
      • rtbgym.envs.simulator.function.WinningPriceDistribution
    • rtbgym.envs.simulator.bidder
      • rtbgym.envs.simulator.bidder.Bidder
    • rtbgym.envs.simulator.rtb_synthetic
      • rtbgym.envs.simulator.rtb_synthetic.RTBSyntheticSimulator
    • rtbgym.utils
      • rtbgym.utils.check_array
      • rtbgym.utils.sigmoid
      • rtbgym.utils.NormalDistribution
  • RECGym Package Reference
    • recgym.envs.rec
      • recgym.envs.rec.RECEnv
    • recgym.envs.simulator.base
      • recgym.envs.simulator.base.BaseUserModel
    • recgym.envs.simulator.function
      • recgym.envs.simulator.function.UserModel
  • BasicGym Package Reference
    • basicgym.envs.synthetic
      • basicgym.envs.synthetic.BasicEnv
    • basicgym.envs.simulator.base
      • basicgym.envs.simulator.base.BaseRewardFunction
      • basicgym.envs.simulator.base.BaseStateTransitionFunction
    • basicgym.envs.simulator.function
      • basicgym.envs.simulator.function.RewardFunction
      • basicgym.envs.simulator.function.StateTransitionFunction

See also:

  • Github
  • LICENSE
  • FAQs
  • News
  • Release Notes
  • Proceedings
  • References
  • SCOPE-RL
  • BasicGym Package Reference

BasicGym Package Reference#

env module#

basicgym.envs.synthetic

Basic Reinforcement Learning (RL) Environment.

simulation module#

basicgym.envs.simulator.base

Abstract Base Class for Simulation.

basicgym.envs.simulator.function

Mathematical Functions used in Synthetic System .

<<< Prev Documentation (Back to Top)

On this page
  • env module
  • simulation module

© Copyright 2023, Haruka Kiyohara, Ren Kishimoto, HAKUHODO Technologies Inc., Hanjuku-kaso Co., Ltd.

Built with the PyData Sphinx Theme 0.13.1.