NuCon (Nucleares Controller)

NuCon is a Python library designed to interface with and control parameters in Nucleares, a nuclear reactor simulation game. It provides a robust, type-safe foundation for reading and writing game parameters, allowing users to easily create their own automations and control systems.

NuCon further provides a work in progress implementation of a reinforcement learning environment for training control policies and a simulator based on model learning.

Note

Nucleares only exposes RODS_POS_ORDERED as writable parameter, and no parameters about core chemistry e.g. Xenon concentration. While NuCon is already usable, it's capabilities are still very limited based on these restrictions. The capabilites are supposed to be extended in future updates to Nucleares, development on the advanced features (Reinforcement / Model Learning) are paused till then.

Features

• Enum-based parameter system for type safety and code clarity
• Support for various parameter types including floats, integers, booleans, strings, and custom enums
• Read and write capabilities for game parameters
• Reinforcement learning environment for training control policies
• Built-in simulator for rapid prototyping and testing
• Model learning for dynamics prediction

Installation

To install NuCon, clone this repository and install via pip:

git clone https://git.dominik-roth.eu/dodox/NuCon
cd NuCon
pip install -e .

Usage

Here's a basic example of how to use NuCon:

from nucon import Nucon

nucon = Nucon()
# or nucon = Nucon(host='localhost', port=8786)

# Enable dummy mode for testing (optional)
nucon.set_dummy_mode(True)

# Read a parameter
core_temp = nucon.CORE_TEMP.value
print(f"Core Temperature: {core_temp}")
# >> Core Temperature: 500.0

# Read a parameter with an enum type
pump_status = nucon.COOLANT_CORE_CIRCULATION_PUMP_0_STATUS.value
print(f"Pump 0 Status: {pump_status}")
if nucon.COOLANT_CORE_CIRCULATION_PUMP_0_STATUS.value:
    print('Pump 0 is active.')
# >> Pump 0 Status: PumpStatus.INACTIVE

# Write to a parameter (has no effect in dummy mode)
nucon.RODS_POS_ORDERED.value = 50
print(f"Rods Position Ordered: {nucon.RODS_POS_ORDERED.value}")
# >> Rods Position Ordered: 50.0

# The repr of an attribute contains all contained info
nucon.CORE_TEMP
# >> NuconParameter(id='CORE_TEMP', value=500.0, param_type=float, writable=False)

API Reference

The nucon instance contains all available parameters.

Parameter properties:

• nucon.<PARAMETER>.value: Get or set the current value of the parameter. Assigning a new value will write it to the game.
• nucon.<PARAMETER>.param_type: Get the type of the parameter
• nucon.<PARAMETER>.is_writable: Check if the parameter is writable
• nucon.<PARAMETER>.enum_type: Get the enum type of the parameter if it's an enum, otherwise None

Parameter methods:

• nucon.<PARAMETER>.read(): Get the current value of the parameter (alias for value)
• nucon.<PARAMETER>.write(new_value, force=False): Write a new value to the parameter. force will try to write even if the parameter is known as non-writable or out of known allowed range.

Class methods:

• nucon.get(parameter): Get the value of a specific parameter. Also accepts string parameter names.
• nucon.set(parameter, value, force=False): Set the value of a specific parameter. Also accepts string parameter names. force will try to write even if the parameter is known as non-writable or out of known allowed range.
• nucon.get_all_readable(): Get a list of all readable parameters (which is all parameters)
• nucon.get_all_writable(): Get a list of all writable parameters
• nucon.get_all(): Get all parameter values as a dictionary
• nucon.get_all_iter(): Get all parameter values as a dictionary as a generator
• nucon.get_multiple(params): Get values for multiple specified parameters
• nucon.get_multiple_iter(params): Get values for multiple specified parameters as a generator
• nucon.set_dummy_mode(dummy_mode): Enable or disable dummy mode for testing. In dummy mode we won't connect to the game and just return sensible values for all params and allow but ignore all writes to writable parameters.

Custom Enum Types:

• PumpStatus: Enum for pump status (INACTIVE, ACTIVE_NO_SPEED_REACHED*, ACTIVE_SPEED_REACHED*, REQUIRES_MAINTENANCE, NOT_INSTALLED, INSUFFICIENT_ENERGY)
• PumpDryStatus: Enum for pump dry status (ACTIVE_WITHOUT_FLUID*, INACTIVE_OR_ACTIVE_WITH_FLUID)
• PumpOverloadStatus: Enum for pump overload status (ACTIVE_AND_OVERLOAD*, INACTIVE_OR_ACTIVE_NO_OVERLOAD)
• BreakerStatus: Enum for breaker status (OPEN*, CLOSED)

*: Truthy value (will be treated as true in e.g. if statements).

So if you're not in the mood to play the game manually, this API can be used to easily create your own automations and control systems. Maybe a little PID controller for the rods? Or, if you wanna go crazy, why not try some

Reinforcement Learning (Work in Progress)

NuCon includes a preliminary Reinforcement Learning (RL) environment based on the OpenAI Gym interface. This allows you to train control policies for the Nucleares game instead of writing them yourself. This feature is currently a work in progress and requires additional dependencies.

Additional Dependencies

To use you'll need to install gymnasium and numpy. You can do so via

pip install -e '.[rl]'

RL Environment

The NuconEnv class in nucon/rl.py provides a Gym-compatible environment for reinforcement learning tasks in the Nucleares simulation. Key features include:

• Observation space: Includes all readable parameters from the NuCon system.
• Action space: Encompasses all writable parameters in the NuCon system.
• Step function: Applies actions to the NuCon system and returns new observations.
• Objective function: Allows for predefined or custom objective functions to be defined for training.

Usage

Here's a basic example of how to use the RL environment:

from nucon.rl import NuconEnv, Parameterized_Objectives

env = NuconEnv(objectives=['max_power'], seconds_per_step=5)
# env2 = gym.make('Nucon-max_power-v0')
# env3 = NuconEnv(objectives=[Parameterized_Objectives['target_temperature'](goal_temp=350)], objective_weights=[1.0], seconds_per_step=5)

obs, info = env.reset()
for _ in range(1000):
    action = env.action_space.sample()  # Your agent here (instead of random)
    obs, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        obs, info = env.reset()
env.close()

Objectives takes either strings of the name of predefined objectives, or lambda functions which take an observation and return a scalar reward. Final rewards are (weighted) summed across all objectives. info['objectives'] contains all objectives and their values.

But theres a problem: RL algorithms require a huge amount of training steps to get passable policies, and Nucleares is a very slow simulation and can not be trivially parallelized. That's why NuCon also provides a

Simulator (Work in Progress)

NuCon provides a built-in simulator to address the challenge of slow training times in the actual Nucleares game. This simulator allows for rapid prototyping and testing of control policies without the need for the full game environment. Key features include:

• Mimics the behavior of the Nucleares game API
• Configurable initial states and operating modes
• Faster than real-time simulation
• Supports parallel execution for increased training throughput

Additional Dependencies

To use you'll need to install torch and flask. You can do so via

pip install -e '.[sim]'

Usage

To use the NuCon simulator:

from nucon import Nucon
from nucon.sim import NuconSimulator, OperatingState

# Create a simulator instance
simulator = NuconSimulator()

# Load a dynamics model (explained later)
simulator.load_model('path/to/model.pth')

# Set initial state (optional)
simulator.set_state(OperatingState.NOMINAL)

# Run the simulator, will start the web server
simulator.run()

# Access via nucon by using the simulator's port
nucon = Nucon(port=simulator.port)

# Or use the simulator with NuconEnv
from nucon.rl import NuconEnv
env = NuconEnv(simulator=simulator) # When given a similator, instead of waiting on the game, we will tell the simulator to skip forward after each step

# Train your RL agent using the simulator
# ...

But theres yet another problem: We do not know the exact simulation dynamics of the game and can therefore not implement an accurate simulator. Thats why NuCon also provides

Model Learning (Work in Progress)

To address the challenge of unknown game dynamics, NuCon provides tools for collecting data, creating datasets, and training models to learn the reactor dynamics. This approach allows for more accurate simulations and enables model-based control strategies. Key features include:

• Data Collection: Supports gathering state transitions from both human play and automated agents.
• Dataset Management: Tools for saving, loading, and merging datasets.
• Model Training: Train neural network models to predict next states based on current states and time deltas.
• Dataset Refinement: Ability to refine datasets by focusing on more challenging or interesting data points.

Additional Dependencies

To use you'll need to install torch and numpy. You can do so via

pip install -e '.[model]'

Usage:

from nucon.model import NuconModelLearner

# Initialize the model learner
learner = NuconModelLearner()

# Collect data by querying the game
learner.collect_data(num_steps=1000)

# Train the model
learner.train_model(batch_size=32, num_epochs=10)

# Refine the dataset
learner.refine_dataset(error_threshold=0.1)

# Save the model and dataset
learner.save_model('reactor_model.pth')
learner.save_dataset('reactor_dataset.pkl')

The trained models can be integrated into the NuconSimulator to provide accurate dynamics based on real game data.

Testing

NuCon includes a test suite to verify its functionality and compatibility with the Nucleares game.

Running Tests

To run the tests:

1. Ensure the Nucleares game is running and accessible at http://localhost:8785/ (or update the URL in the test setup).
2. Install pytest: pip install pytest (or pip install -e .[dev])
3. Run the tests:

   pytest test/test_core.py
   pytest test/test_sim.py
   

Test Coverage

The tests verify:

• Parameter types match their definitions in NuCon
• Writable parameters can be written to
• Non-writable parameters cannot be written to, even when force-writing
• Enum parameters and their custom truthy values behave correctly
• Simulator functionality and consistency

---

To use you'll need to install pillow. You can do so via

pip install -e '.[drake]'

Usage:

from nucon.drake import create_drake_meme

items = [
    (False, "Play Nucleares manually"),
    (True, "Automate it with a script"),
    (False, "But the web interface is tedious to use"),
    (True, "Write an elegant libary to interface with the game and then use that to write the script"),
    (False, "But I would still need to write the control policy by hand"),
    (True, "Let's extend the libary such that it trains a policy via Reinforcement Learning"),
    (False, "But RL takes a huge number of training samples"),
    (True, "Extend the libary to also include an efficient simulator"),
    (False, "But I don't know what the actual internal dynamics are"),
    (True, "Extend the libary once more to also include a neural network dynamics model"),
    (True, "And I'm gonna put a drake meme on the README"),
    (False, "Online meme generators only support a single yes/no pair"),
    (True, "Let's also add a drake meme generator to the libary"),
    ]

meme = create_drake_meme(items)
meme.save("README_meme.jpg")

Disclaimer

NuCon is an unofficial tool and is not affiliated with or endorsed by the creators of Nucleares.

Citing

What? Why would you wanna cite it? What are you even doing?

@misc{nucon,
  title = {NuCon},
  author = {Dominik Roth},
  abstract = {NuCon is a Python library to interface with and control Nucleares, a nuclear reactor simulation game. Includes gymnasium bindings for Reinforcement Learning.},
  url = {https://git.dominik-roth.eu/dodox/NuCon},
  year = {2024},
}