FAQs

SCOPE-RL

Q. xxx environment does not work on SCOPE-RL. How should we fix it?

1. SCOPE-RL is compatible with OpenAI Gym and Gymnasium API, specifically for gym>=0.26.0, which works as follows.

obs, info = env.reset(), False
while not done:
    action = agent.act(obs)
    obs, reward, done, truncated, info = env.step(action)

In contrast, your environment may use the following older interface.

obs = env.reset(), False
while not done:
    action = agent.act(obs)
    obs, reward, done, info = env.step(action)

To solve this incompatibility, please use NewGymAPIWrapper provided in scope_rl/utils.py. It should be used as follows.

from scope_rl.utils import NewGymAPIWrapper
env = NewGymAPIWrapper(env)

Q. xxx environment does not work on d3rlpy, which is used for model training. How should we fix it? (d3rlpy and SCOPE-RL are compatible with a different version of OpenAI Gym.)

1. Both scope-rl>=0.2.1 and d3rlpy>=2.0.2 supports compatibility with gym>=0.26.0 and gymnasium environments. The source is available in the main branch.

If you want to use the older interface of d3rlpy, make sure to use scope-rl==0.1.3 and d3rlpy==1.1.1. Then, please use OldGymAPIWrapper provided in scope_rl/utils.py to enable the use of d3rlpy. The source is available in the depreciated branch.

from scope_rl.utils import OldGymAPIWrapper
env = gym.make("xxx_v0")  # compatible with gym>=0.26.2 and scope-rl==0.1.3
env_ = OldGymAPIWrapper(env)  # compatible with gym<0.26.2 and d3rlpy==1.1.1