Updated README
This commit is contained in:
parent
e665a457dc
commit
c66a4f9e7d
173
README.md
173
README.md
@ -1,17 +1,17 @@
|
|||||||
# NuCon (Nucleares Controller)
|
# NuCon (Nucleares Controller)
|
||||||
|
|
||||||
NuCon is a Python library designed to interface with and control parameters in Nucleares, a nuclear reactor simulation game. It provides a robust, type-safe foundation for reading and writing game parameters, allowing users to easily create their own automations and control systems.
|
NuCon is a Python library designed to interface with and control parameters in [Nucleares](https://store.steampowered.com/app/1428420/Nucleares/), a nuclear reactor simulation game. It provides a robust, type-safe foundation for reading and writing game parameters, allowing users to easily create their own automations and control systems.
|
||||||
|
|
||||||
NuCon further provides a work in progress implementation of a reinforcement learning environment for training control policies.
|
NuCon further provides a work in progress implementation of a reinforcement learning environment for training control policies and a simulator based on model learning.
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- Enum-based parameter system for type safety and code clarity
|
- Enum-based parameter system for type safety and code clarity
|
||||||
- Support for various parameter types including floats, integers, booleans, strings, and custom enums
|
- Support for various parameter types including floats, integers, booleans, strings, and custom enums
|
||||||
- Read and write capabilities for game parameters
|
- Read and write capabilities for game parameters
|
||||||
- Custom truthy values for status enums to simplify conditional logic
|
- Reinforcement learning environment for training control policies
|
||||||
- Dummy mode for testing without connecting to the game
|
- Built-in simulator for rapid prototyping and testing
|
||||||
- Batch operations for getting multiple parameters at once
|
- Model learning for dynamics prediction
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
@ -27,55 +27,54 @@ Here's a basic example of how to use NuCon:
|
|||||||
```python
|
```python
|
||||||
from nucon import Nucon, BreakerStatus
|
from nucon import Nucon, BreakerStatus
|
||||||
|
|
||||||
# Set the base URL for the game's API (if different from default)
|
nucon = Nucon()
|
||||||
Nucon.set_base_url("http://localhost:8785/")
|
# or nucon = Nucon(host='localhost', port=8786)
|
||||||
|
|
||||||
# Enable dummy mode for testing (optional)
|
# Enable dummy mode for testing (optional)
|
||||||
Nucon.set_dummy_mode(True)
|
nucon.set_dummy_mode(True)
|
||||||
|
|
||||||
# Read a parameter
|
# Read a parameter
|
||||||
core_temp = Nucon.CORE_TEMP.value
|
core_temp = nucon.CORE_TEMP.value
|
||||||
print(f"Core Temperature: {core_temp}")
|
print(f"Core Temperature: {core_temp}")
|
||||||
|
|
||||||
# Read a parameter with an enum type
|
# Read a parameter with an enum type
|
||||||
pump_status = Nucon.COOLANT_CORE_CIRCULATION_PUMP_0_STATUS.value
|
pump_status = nucon.COOLANT_CORE_CIRCULATION_PUMP_0_STATUS.value
|
||||||
print(f"Pump 0 Status: {pump_status}")
|
print(f"Pump 0 Status: {pump_status}")
|
||||||
|
|
||||||
# Write to a parameter
|
# Write to a parameter
|
||||||
Nucon.GENERATOR_0_BREAKER.value = BreakerStatus.OPEN # or True
|
nucon.GENERATOR_0_BREAKER.value = BreakerStatus.OPEN # or True
|
||||||
print(f"Generator 0 Breaker Status: {Nucon.GENERATOR_0_BREAKER.value}")
|
print(f"Generator 0 Breaker Status: {nucon.GENERATOR_0_BREAKER.value}")
|
||||||
|
|
||||||
# Use custom truthy values
|
# Use custom truthy values
|
||||||
if Nucon.COOLANT_CORE_CIRCULATION_PUMP_0_STATUS.value:
|
if nucon.COOLANT_CORE_CIRCULATION_PUMP_0_STATUS.value:
|
||||||
print("Pump 0 is active")
|
print("Pump 0 is active")
|
||||||
```
|
```
|
||||||
|
|
||||||
## API Reference
|
## API Reference
|
||||||
|
|
||||||
The `Nucon` enum contains all available parameters. Each parameter is defined with:
|
The `nucon` instance contains all available parameters. Each parameter is defined with:
|
||||||
- An ID (string)
|
- An ID (string)
|
||||||
- A type (float, int, bool, str, or a custom Enum)
|
- A type (float, int, bool, str, or a custom Enum)
|
||||||
- A boolean indicating whether it's writable
|
- A boolean indicating whether it's writable
|
||||||
|
|
||||||
Parameter properties:
|
Parameter properties:
|
||||||
- `Nucon.<PARAMETER>.value`: Get or set the current value of the parameter. Assigning a new value will write it to the game.
|
- `nucon.<PARAMETER>.value`: Get or set the current value of the parameter. Assigning a new value will write it to the game.
|
||||||
- `Nucon.<PARAMETER>.param_type`: Get the type of the parameter
|
- `nucon.<PARAMETER>.param_type`: Get the type of the parameter
|
||||||
- `Nucon.<PARAMETER>.is_writable`: Check if the parameter is writable
|
- `nucon.<PARAMETER>.is_writable`: Check if the parameter is writable
|
||||||
- `Nucon.<PARAMETER>.enum_type`: Get the enum type of the parameter if it's an enum, otherwise None
|
- `nucon.<PARAMETER>.enum_type`: Get the enum type of the parameter if it's an enum, otherwise None
|
||||||
|
|
||||||
Parameter methods:
|
Parameter methods:
|
||||||
- `Nucon.<PARAMETER>.read()`: Get the current value of the parameter (alias for `value`)
|
- `nucon.<PARAMETER>.read()`: Get the current value of the parameter (alias for `value`)
|
||||||
- `Nucon.<PARAMETER>.write(new_value, force=False)`: Write a new value to the parameter. `force` will try to write even if the parameter is known as non-writable or out of known allowed range.
|
- `nucon.<PARAMETER>.write(new_value, force=False)`: Write a new value to the parameter. `force` will try to write even if the parameter is known as non-writable or out of known allowed range.
|
||||||
|
|
||||||
Class methods:
|
Class methods:
|
||||||
- `Nucon.get(parameter)`: Get the value of a specific parameter. Also accepts string parameter names.
|
- `nucon.get(parameter)`: Get the value of a specific parameter. Also accepts string parameter names.
|
||||||
- `Nucon.set(parameter, value, force=False)`: Set the value of a specific parameter. Also accepts string parameter names. `force` will try to write even if the parameter is known as non-writable or out of known allowed range.
|
- `nucon.set(parameter, value, force=False)`: Set the value of a specific parameter. Also accepts string parameter names. `force` will try to write even if the parameter is known as non-writable or out of known allowed range.
|
||||||
- `Nucon.get_all_readable()`: Get a list of all readable parameters (which is all parameters)
|
- `nucon.get_all_readable()`: Get a list of all readable parameters (which is all parameters)
|
||||||
- `Nucon.get_all_writable()`: Get a list of all writable parameters
|
- `nucon.get_all_writable()`: Get a list of all writable parameters
|
||||||
- `Nucon.get_all()`: Get all parameter values as a dictionary
|
- `nucon.get_all()`: Get all parameter values as a dictionary
|
||||||
- `Nucon.get_multiple(params)`: Get values for multiple specified parameters
|
- `nucon.get_multiple(params)`: Get values for multiple specified parameters
|
||||||
- `Nucon.set_base_url(url)`: Set the base URL for the game's API
|
- `nucon.set_dummy_mode(dummy_mode)`: Enable or disable dummy mode for testing
|
||||||
- `Nucon.set_dummy_mode(dummy_mode)`: Enable or disable dummy mode for testing
|
|
||||||
|
|
||||||
Custom Enum Types:
|
Custom Enum Types:
|
||||||
- `PumpStatus`: Enum for pump status (INACTIVE, ACTIVE_NO_SPEED_REACHED, ACTIVE_SPEED_REACHED, REQUIRES_MAINTENANCE, NOT_INSTALLED, INSUFFICIENT_ENERGY)
|
- `PumpStatus`: Enum for pump status (INACTIVE, ACTIVE_NO_SPEED_REACHED, ACTIVE_SPEED_REACHED, REQUIRES_MAINTENANCE, NOT_INSTALLED, INSUFFICIENT_ENERGY)
|
||||||
@ -83,13 +82,15 @@ Custom Enum Types:
|
|||||||
- `PumpOverloadStatus`: Enum for pump overload status (ACTIVE_AND_OVERLOAD, INACTIVE_OR_ACTIVE_NO_OVERLOAD)
|
- `PumpOverloadStatus`: Enum for pump overload status (ACTIVE_AND_OVERLOAD, INACTIVE_OR_ACTIVE_NO_OVERLOAD)
|
||||||
- `BreakerStatus`: Enum for breaker status (OPEN, CLOSED)
|
- `BreakerStatus`: Enum for breaker status (OPEN, CLOSED)
|
||||||
|
|
||||||
|
So if you're not in the mood to play the game manually, this API can be used to easily create your own automations and control systems. Maybe a little PID controller for the rods? Or, if you wanna go crazy, why not try some
|
||||||
|
|
||||||
## Reinforcement Learning (Work in Progress)
|
## Reinforcement Learning (Work in Progress)
|
||||||
|
|
||||||
NuCon includes a preliminary Reinforcement Learning (RL) environment based on the OpenAI Gym interface. This feature is currently a work in progress and requires additional dependencies.
|
NuCon includes a preliminary Reinforcement Learning (RL) environment based on the OpenAI Gym interface. This allows you to train control policies for the Nucleares game instead of writing them yourself. This feature is currently a work in progress and requires additional dependencies.
|
||||||
|
|
||||||
### Additional Dependencies
|
### Additional Dependencies
|
||||||
|
|
||||||
To use the RL features, you'll need to install the following packages:
|
To use you'll need to install the following packages:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install gymnasium numpy
|
pip install gymnasium numpy
|
||||||
@ -99,9 +100,9 @@ pip install gymnasium numpy
|
|||||||
|
|
||||||
The `NuconEnv` class in `nucon/rl.py` provides a Gym-compatible environment for reinforcement learning tasks in the Nucleares simulation. Key features include:
|
The `NuconEnv` class in `nucon/rl.py` provides a Gym-compatible environment for reinforcement learning tasks in the Nucleares simulation. Key features include:
|
||||||
|
|
||||||
- Observation space: Includes all readable parameters from the Nucon system.
|
- Observation space: Includes all readable parameters from the NuCon system.
|
||||||
- Action space: Encompasses all writable parameters in the Nucon system.
|
- Action space: Encompasses all writable parameters in the NuCon system.
|
||||||
- Step function: Applies actions to the Nucon system and returns new observations.
|
- Step function: Applies actions to the NuCon system and returns new observations.
|
||||||
- Objective function: Allows for predefined or custom objective functions to be defined for training.
|
- Objective function: Allows for predefined or custom objective functions to be defined for training.
|
||||||
|
|
||||||
### Usage
|
### Usage
|
||||||
@ -112,7 +113,7 @@ from nucon.rl import NuconEnv, Parameterized_Objectives
|
|||||||
|
|
||||||
env = NuconEnv(objectives=['max_power'], seconds_per_step=5)
|
env = NuconEnv(objectives=['max_power'], seconds_per_step=5)
|
||||||
# env2 = gym.make('Nucon-max_power-v0')
|
# env2 = gym.make('Nucon-max_power-v0')
|
||||||
# env3 = NuconEnv(objectives=[Parameterized_Objectives['target_temperature'](goal_temp=600)], objective_weights=[1.0], seconds_per_step=5)
|
# env3 = NuconEnv(objectives=[Parameterized_Objectives['target_temperature'](goal_temp=350)], objective_weights=[1.0], seconds_per_step=5)
|
||||||
|
|
||||||
obs, info = env.reset()
|
obs, info = env.reset()
|
||||||
for _ in range(1000):
|
for _ in range(1000):
|
||||||
@ -126,6 +127,99 @@ env.close()
|
|||||||
|
|
||||||
Objectives takes either strings of the name of predefined objectives, or lambda functions which take an observation and return a scalar reward. Final rewards are (weighted) summed across all objectives. `info['objectives']` contains all objectives and their values.
|
Objectives takes either strings of the name of predefined objectives, or lambda functions which take an observation and return a scalar reward. Final rewards are (weighted) summed across all objectives. `info['objectives']` contains all objectives and their values.
|
||||||
|
|
||||||
|
But theres a problem: RL algorithms require a huge amount of training steps to get passable policies, and Nucleares is a very slow simulation and can not be trivially parallelized. That's why NuCon also provides a
|
||||||
|
|
||||||
|
## Simulator (Work in Progress)
|
||||||
|
|
||||||
|
NuCon provides a built-in simulator to address the challenge of slow training times in the actual Nucleares game. This simulator allows for rapid prototyping and testing of control policies without the need for the full game environment. Key features include:
|
||||||
|
|
||||||
|
- Mimics the behavior of the Nucleares game API
|
||||||
|
- Configurable initial states and operating modes
|
||||||
|
- Faster than real-time simulation
|
||||||
|
- Supports parallel execution for increased training throughput
|
||||||
|
|
||||||
|
### Additional Dependencies
|
||||||
|
|
||||||
|
To use you'll need to install the following packages:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install torch flask
|
||||||
|
```
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
|
||||||
|
To use the NuCon simulator:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nucon import Nucon
|
||||||
|
from nucon.sim import NuconSimulator, OperatingState
|
||||||
|
|
||||||
|
# Create a simulator instance
|
||||||
|
simulator = NuconSimulator()
|
||||||
|
|
||||||
|
# Load a dynamics model (explained later)
|
||||||
|
simulator.load_model('path/to/model.pth')
|
||||||
|
|
||||||
|
# Set initial state (optional)
|
||||||
|
simulator.set_state(OperatingState.NOMINAL)
|
||||||
|
|
||||||
|
# Run the simulator, will start the web server
|
||||||
|
simulator.run()
|
||||||
|
|
||||||
|
# Access via nucon by using the simulator's port
|
||||||
|
nucon = Nucon(port=simulator.port)
|
||||||
|
|
||||||
|
# Or use the simulator with NuconEnv
|
||||||
|
from nucon.rl import NuconEnv
|
||||||
|
env = NuconEnv(simulator=simulator) # When given a similator, instead of waiting on the game, we will tell the simulator to skip forward after each step
|
||||||
|
|
||||||
|
# Train your RL agent using the simulator
|
||||||
|
# ...
|
||||||
|
```
|
||||||
|
|
||||||
|
But theres yet another problem: We do not know the exact simulation dynamics of the game and can therefore not implement an accurate simulator. Thats why NuCon also provides
|
||||||
|
|
||||||
|
## Model Learning (Work in Progress)
|
||||||
|
|
||||||
|
To address the challenge of unknown game dynamics, NuCon provides tools for collecting data, creating datasets, and training models to learn the reactor dynamics. This approach allows for more accurate simulations and enables model-based control strategies. Key features include:
|
||||||
|
|
||||||
|
- Data Collection: Supports gathering state transitions from both human play and automated agents.
|
||||||
|
- Dataset Management: Tools for saving, loading, and merging datasets.
|
||||||
|
- Model Training: Train neural network models to predict next states based on current states and time deltas.
|
||||||
|
- Dataset Refinement: Ability to refine datasets by focusing on more challenging or interesting data points.
|
||||||
|
|
||||||
|
### Additional Dependencies
|
||||||
|
|
||||||
|
To use the RL features, you'll need to install the following packages:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install torch numpy
|
||||||
|
```
|
||||||
|
|
||||||
|
### Usage:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nucon.model import NuconModelLearner
|
||||||
|
|
||||||
|
# Initialize the model learner
|
||||||
|
learner = NuconModelLearner()
|
||||||
|
|
||||||
|
# Collect data by querying the game
|
||||||
|
learner.collect_data(num_steps=1000)
|
||||||
|
|
||||||
|
# Train the model
|
||||||
|
learner.train_model(batch_size=32, num_epochs=10)
|
||||||
|
|
||||||
|
# Refine the dataset
|
||||||
|
learner.refine_dataset(error_threshold=0.1)
|
||||||
|
|
||||||
|
# Save the model and dataset
|
||||||
|
learner.save_model('reactor_model.pth')
|
||||||
|
learner.save_dataset('reactor_dataset.pkl')
|
||||||
|
```
|
||||||
|
|
||||||
|
The trained models can be integrated into the NuconSimulator to provide accurate dynamics based on real game data.
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
NuCon includes a test suite to verify its functionality and compatibility with the Nucleares game.
|
NuCon includes a test suite to verify its functionality and compatibility with the Nucleares game.
|
||||||
@ -136,7 +230,11 @@ To run the tests:
|
|||||||
|
|
||||||
1. Ensure the Nucleares game is running and accessible at http://localhost:8785/ (or update the URL in the test setup).
|
1. Ensure the Nucleares game is running and accessible at http://localhost:8785/ (or update the URL in the test setup).
|
||||||
2. Install pytest: `pip install pytest`
|
2. Install pytest: `pip install pytest`
|
||||||
3. Run the tests: `pytest test/test.py`
|
3. Run the tests:
|
||||||
|
```bash
|
||||||
|
pytest test/test_core.py
|
||||||
|
pytest test/test_sim.py
|
||||||
|
```
|
||||||
|
|
||||||
### Test Coverage
|
### Test Coverage
|
||||||
|
|
||||||
@ -146,6 +244,7 @@ The tests verify:
|
|||||||
- Writable parameters can be written to
|
- Writable parameters can be written to
|
||||||
- Non-writable parameters cannot be written to, even when force-writing
|
- Non-writable parameters cannot be written to, even when force-writing
|
||||||
- Enum parameters and their custom truthy values behave correctly
|
- Enum parameters and their custom truthy values behave correctly
|
||||||
|
- Simulator functionality and consistency
|
||||||
|
|
||||||
## Disclaimer
|
## Disclaimer
|
||||||
NuCon is an unofficial tool and is not affiliated with or endorsed by the creators of Nucleares.
|
NuCon is an unofficial tool and is not affiliated with or endorsed by the creators of Nucleares.
|
||||||
@ -156,8 +255,8 @@ What? Why would you wanna cite it? What are you even doing?
|
|||||||
@misc{nucon,
|
@misc{nucon,
|
||||||
title = {NuCon},
|
title = {NuCon},
|
||||||
author = {Dominik Roth},
|
author = {Dominik Roth},
|
||||||
abstract = {NuCon is a Python library to interface with and control Nucleares, a nuclear reactor simulation game. Includes gymnasium bindings for Reinforcement Learning.}
|
abstract = {NuCon is a Python library to interface with and control Nucleares, a nuclear reactor simulation game. Includes gymnasium bindings for Reinforcement Learning.},
|
||||||
url = {https://git.dominik-roth.eu/dodox/NuCon},
|
url = {https://git.dominik-roth.eu/dodox/NuCon},
|
||||||
year = {2024},
|
year = {2024},
|
||||||
}
|
}
|
||||||
```
|
```
|
Loading…
Reference in New Issue
Block a user