Setting Up A Gym

Building on top of the previous tutorial:

In this tutorial, we walk through how to interact with the FrankaEnv environment from Ark. This interface is built to resemble OpenAI Gym, enabling easy integration with reinforcement learning and control pipelines.

🛠️ Prerequisites

Before starting, ensure:

• You have a Franka robot set up or simulation capabilities.

• The required FrankaEnv module is available.

• You have a valid configuration YAML file (e.g., config/global_config.yaml).

🚀 Code Walkthrough

1. Import Required Modules

from scripts.franka_env import FrankaEnv
import time

• FrankaEnv: The main environment class that wraps the Franka robot or its simulation.

• time: Used to introduce delays if needed (not mandatory for basic control).

2. Define Configuration Parameters

SIM = True  # Set to False if you want to use the real robot
CONFIG = 'config/global_config.yaml'

• SIM: Flag to toggle between simulation (True) and real robot (False).

• CONFIG: Path to the YAML config that defines robot behavior, controller settings, etc.

3. Initialise the Environment

env = FrankaEnv(sim=SIM, config=CONFIG)

This creates an instance of the environment. Internally, it:

• Loads the robot model or connects to the real hardware.

• Sets up communication channels and state observation.

• Loads parameters from the YAML file.

4. Reset the Environment

observation, info = env.reset()

• Resets the simulation or real robot to a known starting state.

• Returns:
  • observation: Initial state of the robot.
  • info: Optional debug or metadata.

5. Define a Policy

def policy(observation):
    ...
    return action

This is a placeholder for your control policy. It takes in the current observation and returns an action:

• Could be a hard-coded rule, inverse kinematics output, or a learned neural policy.

• Example:

  def policy(obs):
      return [random.uniform(-3, 3) for _ in range(9)]

6. Run the Control Loop

for _ in range(1000):
    action = policy(observation)
    observation, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        observation, info = env.reset()

• Loop: Runs the robot environment for 1000 timesteps.

• env.step(action):
  • Sends the action to the robot or sim.
  • Returns:
    • observation: Next state.
    • reward: Optional scalar for RL use.
    • terminated: Episode ended normally.
    • truncated: Episode was cut off due to time or safety.
    • info: Extra diagnostics.

• Reset on end: If an episode ends, reset the environment.

Running the Gym

You need to run two nodes the simulation:

• The Simulation Node

• The Gym Node

You should see the Franka move randomly or according to the policy you have defined.

Code can be found:

Next Steps

Try writing your own Behavioural Cloning Algorithm, Diffusion Policy or even ACT!