100 Commits
Author SHA1 Message Date
dodox 99769e11b3 Fix references in confifgs were resolved from wrong experiment 2024-08-22 20:13:34 +02:00
dodox f14e877fbd Fix: broken args when comitting exps to slurm 2024-08-22 17:47:12 +02:00
dodox 24b232138f Fixed bug with how configs are merged 2024-08-16 12:54:29 +02:00
dodox 3b138bba1e Bug fixes + tests 2024-08-16 12:39:57 +02:00
dodox f2b5837b56 Fix duplicate func definition 2024-08-15 19:07:35 +02:00
dodox 62a5b63d89 . 2024-08-15 19:03:56 +02:00
dodox ebca76e680 Maybe better cli interface und ressource allocation 2024-08-15 18:54:36 +02:00
dodox 69c79bdb6f Added TODO to README 2024-06-29 19:35:29 +02:00
dodox ff1ba0be03 . 2024-06-24 20:42:02 +02:00
dodox 1830c36fd8 allow running in isaac singularity container 2024-06-24 18:23:20 +02:00
dodox ea3d3b4b82 No warning on unconsumed grid, ablative, vars 2024-04-04 15:13:35 +02:00
dodox 07817ede2a Ensure name and project are consumed for slurm launches 2024-02-16 14:01:02 +01:00
dodox 2d0e552c6c BugFix: Error when printing unconsumed configs 2024-02-16 13:59:17 +01:00
dodox 65350ade74 better warning about uncosnumed configs 2024-02-16 13:57:48 +01:00
dodox 6c29d49a33 Pseudo name for pre validation 2024-02-10 19:43:58 +01:00
dodox 3d8af45c7c Tell Runners their current name on setup 2024-02-10 19:41:24 +01:00
dodox 9762e50987 Support for xvfb-run wrapper 2024-02-02 18:16:46 +01:00
dodox 03d2ab1733 DOnt retry if there was no error 2024-01-16 15:48:20 +01:00
dodox d812469c7c Retry after wandb CommError 2023-09-18 17:16:22 +02:00
dodox e7d6326546 Fix: Ensure job_type never exceeds limit 2023-09-07 17:12:49 +02:00
dodox ff5b79feaf Bugfix: incorrect access to bind_to_core arg 2023-09-02 21:07:50 +02:00
dodox bc698fec54 Allow manual verification before job submission 2023-09-02 20:54:53 +02:00
dodox 695046a123 Support binding agent to core 2023-09-02 19:55:35 +02:00
dodox ed5081f1b8 Fix: Typo in sh_lines for slurm 2023-08-01 11:51:38 +02:00
dodox ee83a3be52 Allow access to run_id from config 2023-07-31 15:38:46 +02:00
dodox 6f9fbe6b28 Seperate job_id and task_id and allow access from configs 2023-07-31 15:34:10 +02:00
dodox a16f3889f6 {tmp} contains expanded path of $TMP 2023-07-30 13:39:41 +02:00
dodox 25e64774bb . 2023-07-29 22:04:54 +02:00
dodox 7ed0c8fcc5 Fixed bug: Jobs working on wrong reps 2023-07-29 14:53:19 +02:00
dodox 0d7b02a57b [WIP] debugging 2023-07-29 14:45:32 +02:00
dodox 9260ea01f3 Allow configuring reps per version 2023-07-29 14:31:49 +02:00
dodox 94408f6b08 Added examples fro grid/abl to example.yaml 2023-07-29 13:03:24 +02:00
dodox 2248b24bcb [WIP] Implemented Grid and ablative
(functional but ugly)
2023-07-29 13:03:01 +02:00
dodox 74b06d92e7 ChatGPT lied (Revert "CHatGPT tells me i need to init BEFORE I sweep")
This reverts commit 373346b589.
2023-07-29 11:53:29 +02:00
dodox 373346b589 CHatGPT tells me i need to init BEFORE I sweep 2023-07-27 16:38:24 +02:00
dodox f484e9d24c fix: need to pass to settings as kwargs 2023-07-27 15:10:52 +02:00
dodox 98ce730086 Trying to work around wandb quirks 2023-07-27 14:39:05 +02:00
dodox d4ac0ac275 typo 2023-07-27 13:19:29 +02:00
dodox e9702cd6a0 fix: sweep_id not passed correctly 2023-07-27 13:17:21 +02:00
dodox c1ba7e01e8 Revert "wip"
This reverts commit 2058ee4932.
2023-07-27 13:13:48 +02:00
dodox 2058ee4932 wip 2023-07-27 13:09:34 +02:00
dodox 15381955fa ugly fix: dont traverse dots in sweep config 2023-07-27 13:07:18 +02:00
dodox e926714275 sweep also takes no settings 2023-07-27 12:54:17 +02:00
dodox 4b51557501 fix: sweep takes no reinit arg 2023-07-27 12:52:25 +02:00
dodox 160def65b2 more fixes for sweeps 2023-07-27 12:50:38 +02:00
dodox b7634384f5 minor fix 2023-07-27 12:49:30 +02:00
dodox 6753119c1d Revert "minor fix"
This reverts commit 127daa30c5.
2023-07-27 12:42:02 +02:00
dodox 127daa30c5 minor fix 2023-07-27 12:40:36 +02:00
dodox c29a109175 fixed name collission frro args 2023-07-27 12:37:59 +02:00
dodox cb4537e5b9 Only init sweep once (on login-node for slurm) 2023-07-27 12:34:36 +02:00
dodox 966dfbcfb8 minor refactor 2023-07-27 11:33:45 +02:00
dodox 72d28de17b Make more wandb options configurable 2023-07-27 11:29:06 +02:00
dodox eddc5e6092 supply permuted config to wandb on sweep 2023-07-12 18:20:21 +02:00
dodox 47c0b7b2c8 maybe fix wandb sweeps 2023-07-12 18:02:10 +02:00
dodox df0ca53a0a 'export' Slate_Runner 2023-07-12 13:08:01 +02:00
dodox 6febf19800 Allow pre validating runners before submitting to slurm 2023-07-12 13:06:14 +02:00
dodox 97137964fb pdb runner 2023-07-12 12:23:18 +02:00
dodox e61bf250a5 fix 2023-07-12 11:49:30 +02:00
dodox cb8a454639 fixing bugs 2023-07-12 11:46:52 +02:00
dodox 72a7b9df25 Append to job_log 2023-07-12 11:38:58 +02:00
dodox 10b479d8d0 fix: typo 2023-07-12 11:15:25 +02:00
dodox df9118e531 more config printing 2023-07-12 11:14:22 +02:00
dodox c4089c6ea4 fix config printer 2023-07-12 11:13:58 +02:00
dodox eac7244de9 Merge branch 'master' of dominik-roth.eu:dodox/Slate 2023-07-12 11:13:03 +02:00
dodox 0a3205fca1 pretty printing 2023-07-12 11:12:18 +02:00
dodox fb3fece928 pretty printing 2023-07-12 11:11:27 +02:00
dodox e1af07a68d Fix expand vars in array 2023-07-12 11:07:33 +02:00
dodox ee2efa6da9 wandb 2023-07-10 11:26:46 +02:00
dodox d516ac9888 WANDB 2023-07-10 11:22:20 +02:00
dodox be837df844 wandb 2023-07-10 11:16:55 +02:00
dodox a1619bcd04 wandb fixes 2023-07-10 10:57:50 +02:00
dodox 4efa081d73 wandb tweaks 2023-07-09 18:06:06 +02:00
dodox a9bde894b8 . 2023-07-09 17:53:03 +02:00
dodox 733286e810 bwuni is annoying 2023-07-09 17:47:06 +02:00
dodox 4e5f10378a fix: put wandb into seperate thread 2023-07-09 17:41:41 +02:00
dodox 7dbb3b32be sweep bug fixed 2023-07-09 17:37:05 +02:00
dodox 51fb6843c2 smashing bugs 2023-07-09 17:25:57 +02:00
dodox 02564de48b ... 2023-07-09 17:19:37 +02:00
dodox 303d0f2dfe . 2023-07-09 17:16:50 +02:00
dodox 4a04480105 buuuug 2023-07-09 17:15:39 +02:00
dodox f2953cd80a bugfix 2023-07-09 17:04:57 +02:00
dodox 84af1865b8 bugs away 2023-07-09 16:52:09 +02:00
dodox 93b7d68e31 More logging 2023-07-09 16:44:09 +02:00
dodox b5e5664cbc more bugfixes 2023-07-09 16:31:35 +02:00
dodox fa570617d7 minor fixes 2023-07-09 16:23:13 +02:00
dodox fd43bbabb6 Make sure we consume the conf instead of just reading it 2023-07-09 16:21:33 +02:00
dodox 7bc7f319d8 removed srand 2023-07-09 16:18:47 +02:00
dodox b0a8421155 fix: typo 2023-07-09 16:18:03 +02:00
dodox ee70a5239c fix: Exception when consuming dict 2023-07-09 16:16:24 +02:00
dodox ebe4e04861 Support threading 2023-07-09 16:12:38 +02:00
dodox 756c096f56 Inform user about unconsumed config 2023-07-08 12:52:59 +02:00
dodox db5b9ac399 Trying to fix bugs with config uploaded to wandb 2023-07-08 11:48:21 +02:00
dodox 0090d502f9 Make config consumed checks optional 2023-07-07 16:40:30 +02:00
dodox cdb1dad534 Little hack to cast str to int for rand seed 2023-07-07 16:02:45 +02:00
dodox d39877b7fb Give configs access to random numbers 2023-07-07 15:46:56 +02:00
dodox 73b974458b fix: job_id is an int 2023-07-07 15:27:16 +02:00
dodox 5e8d6d2552 Multiple Processes fro Slurm 2023-07-07 14:39:38 +02:00
dodox be76a5363a Implemented Process Pool 2023-07-07 13:10:06 +02:00
dodox a8374d864c fix: build script suing newlines 2023-07-07 12:38:07 +02:00
dodox a663d7e9d4 fix: invalid batch file 2023-07-07 12:32:12 +02:00