Zhangzheng/CGA-bench
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
CGA-bench/saves/0420~0426/MyExperimentsSimple/SimpleSPI_20260421_214748/SimpleSPI_20260421_214748.log

363 lines
18 KiB
Plaintext
Raw Permalink Normal View History

first commit
2026-05-22 10:02:42 +08:00
---------------custom config--------------
run:
mode: autoline
save:
en: True
pub:
prefix: SimpleSPI
subdir: MyExperimentsSimple
gpt:
model: qwen-max
rtlgen_model: qwen-max
autoline:
result_path: results/myproject_simple_spi
cga:
enabled: True
max_iter: 15
target_coverage: 85.0
probset:
path: data/myproject/spi_controller_simple.jsonl
mutant_path: None
more_info_paths: []
only: ['spi_controller_simple']
promptscript: pychecker
timeout: 300
save_compile: False
debug:
max: 3
itermax: 10
update_desc: False
TBcheck:
discrim_mode: col_70_wrong_row_25_correct
------------------------------------------
------config info (custom + default)------
run:
version: 2.0
author: Ruidi Qiu - Technical University of Munich
time: 20260421_214748
custom_path: config/myproject_simple_spi.yaml
mode: autoline
hostname: localhost
pid: 318145
pyversion: 3.12.3 (main, Mar 3 2026, 12:15:18) [GCC 13.3.0]
save:
en: True
root: saves/0420~0426/MyExperimentsSimple/SimpleSPI_20260421_214748/
pub:
prefix: SimpleSPI
dir: saves/0420~0426/
subdir: MyExperimentsSimple/
log:
en: True
dir: logs/
notes: None
cfg_pmode: iwantall
debug_en: False
level: TRACE
message:
en: True
dir: messages/
format: json
iverilog:
en: True
subdir: ivcode_nodebug
load:
prompt:
path: config/initial_prompts/prompt1.txt
pick_idx: []
stage_template:
path: config/templates/stage_template0301.txt
gpt:
model: qwen-max
key_path: config/key_API.json
temperature: None
json_mode: False
chatgpt:
start_form: chat
one_time_talk: False
rtlgen_model: qwen-max
iverilog:
dir:
task_id:
autoline:
result_path: results/myproject_simple_spi
cga:
enabled: True
max_iter: 15
target_coverage: 85.0
probset:
path: data/myproject/spi_controller_simple.jsonl
mutant_path: None
gptgenRTL_path: None
more_info_paths: []
only: ['spi_controller_simple']
exclude: []
exclude_json: None
filter: [{}]
checklist:
max: 3
debug:
max: 3
reboot: 1
py_rollback: 2
onlyrun: None
promptscript: pychecker
timeout: 300
TBcheck:
rtl_num: 20
correct_max: 3
discrim_mode: col_70_wrong_row_25_correct
correct_mode: naive
rtl_compens_en: True
rtl_compens_max_iter: 3
itermax: 10
update_desc: False
save_compile: False
save_finalcodes: True
error_interruption: False
stage3:
rtl_mode: auto
max_inline_chars: 5000
save_rtl_file: True
multi_tb:
enabled: False
auto_threshold_lines: 500
strategy: functional
max_tb_count: 5
parallel: False
merge_coverage: True
_initialized: True
------------------------------------------
--------------default config--------------
run:
version: 2.0
author: Ruidi Qiu - Technical University of Munich
time: None
custom_path: None
mode: qwen-max
save:
en: True
root: None
pub:
prefix: None
dir: saves/$weekrange$/
subdir:
log:
en: True
dir: logs/
notes: None
cfg_pmode: iwantall
debug_en: False
level: TRACE
message:
en: True
dir: messages/
format: json
iverilog:
en: True
subdir: ivcode_nodebug
load:
prompt:
path: config/initial_prompts/prompt1.txt
pick_idx: []
stage_template:
path: config/templates/stage_template0301.txt
gpt:
model: 4o
key_path: config/key_API.json
temperature: None
json_mode: False
chatgpt:
start_form: chat
one_time_talk: False
rtlgen_model: None
iverilog:
dir:
task_id:
autoline:
result_path: results
cga:
enabled: True
max_iter: 10
target_coverage: 100.0
probset:
path: None
mutant_path: None
gptgenRTL_path: None
more_info_paths: []
only: ['review2015_fancytimer', 'fsm_ps2data', 'bugs_case', 'review2015_fsmonehot', 'review2015_fsmseq', 'lemmings4', 'ece241_2013_q8']
exclude: []
exclude_json: None
filter: [{}]
checklist:
max: 3
debug:
max: 5
reboot: 1
py_rollback: 2
onlyrun: None
promptscript: None
timeout: 300
TBcheck:
rtl_num: 20
correct_max: 3
discrim_mode: col_full_wrong
correct_mode: naive
rtl_compens_en: True
rtl_compens_max_iter: 3
itermax: 10
update_desc: False
save_compile: True
save_finalcodes: True
error_interruption: False
stage3:
rtl_mode: auto
max_inline_chars: 5000
save_rtl_file: True
multi_tb:
enabled: False
auto_threshold_lines: 500
strategy: functional
max_tb_count: 5
parallel: False
merge_coverage: True
------------------------------------------
2026-04-21 21:47:48 | INFO | all configurations are loaded, starting the main process...
2026-04-21 21:47:48 | INFO |
2026-04-21 21:47:48 | INFO | ######################### task 1/1 [spi_controller_simple] #########################
2026-04-21 21:48:28 | INFO | [spi_controller_simple] [TBgen] stage_0 ends (40.00s used)
2026-04-21 21:49:10 | INFO | [spi_controller_simple] [TBgen] stage_1 ends (41.89s used)
2026-04-21 21:50:13 | INFO | [spi_controller_simple] [TBgen] stage_2 ends (62.72s used)
2026-04-21 21:51:13 | INFO | [spi_controller_simple] [TBgen] stage_3 ends (59.93s used)
2026-04-21 21:54:04 | INFO | [spi_controller_simple] [TBgen] stage_4 ends (170.75s used)
2026-04-21 21:54:04 | INFO | [spi_controller_simple] [TBgen] stage_checklist ends (0.00s used)
2026-04-21 23:13:10 | INFO | [spi_controller_simple] [TBgen] stage_4b ends (4746.37s used)
2026-04-21 23:13:54 | INFO | [spi_controller_simple] [TBgen] stage_5 ends (44.31s used)
2026-04-21 23:13:54 | INFO | [spi_controller_simple]
2026-04-21 23:13:54 | INFO | [spi_controller_simple] [TBsim] iverilog compilation : passed!
2026-04-21 23:13:55 | ERROR | [spi_controller_simple] Error when running TBsim, iter: 1. Message: [Errno 2] No such file or directory: 'saves/0420~0426/MyExperimentsSimple/SimpleSPI_20260421_214748/spi_controller_simple/1_1_TBgen/TBgen_codes/TBout.txt' | location: autoline/TB_autoline.py, func: run_stages_core, line: 389 | caller: location: autoline/TB_autoline.py, func: run_stages, line: 318
2026-04-21 23:13:55 | WARNING | [spi_controller_simple] ⚠️ Pipeline interrupted. Cooling down for 15s to avoid API Rate Limit... | location: autoline/TB_autoline.py, func: run_stages_core, line: 394 | caller: location: autoline/TB_autoline.py, func: run_stages, line: 318
2026-04-21 23:14:41 | INFO | [spi_controller_simple] [TBgen] stage_0 ends (31.82s used)
2026-04-21 23:15:40 | INFO | [spi_controller_simple] [TBgen] stage_1 ends (58.45s used)
2026-04-21 23:16:20 | INFO | [spi_controller_simple] [TBgen] stage_2 ends (40.68s used)
2026-04-21 23:17:19 | INFO | [spi_controller_simple] [TBgen] stage_3 ends (58.39s used)
2026-04-21 23:19:31 | INFO | [spi_controller_simple] [TBgen] stage_4 ends (131.84s used)
2026-04-21 23:19:31 | INFO | [spi_controller_simple] [TBgen] stage_checklist ends (0.00s used)
2026-04-22 00:00:07 | INFO | [spi_controller_simple] [TBgen] stage_4b ends (2436.44s used)
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBgen] stage_5 ends (46.39s used)
2026-04-22 00:00:54 | INFO | [spi_controller_simple]
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBsim] iverilog compilation : passed!
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBsim] python simulation : passed!
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBsim] TBsim finished : True!
2026-04-22 00:00:54 | INFO | [spi_controller_simple]
2026-04-22 00:00:54 | INFO | [spi_controller_simple] rtl list not found, generating naive rtls for testbench checking
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: single_bit_flip | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: swap_shift | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: single_bit_flip | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: single_bit_flip | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: tiny_offset | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: invert_single_signal | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: wrong_bit_index | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: tiny_offset | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: tiny_offset | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: tiny_offset | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: tiny_offset | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: tiny_offset | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: invert_single_signal | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: invert_single_signal | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: invert_single_signal | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: single_bit_flip | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: invert_single_signal | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: tiny_offset | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: single_bit_flip | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | DEBUG | [spi_controller_simple] Applied mutation: invert_counter | location: autoline/rtl_mutator.py, func: _safe_log, line: 33
2026-04-22 00:00:54 | INFO | [spi_controller_simple] 20 mutation-based RTLs generated (from reference RTL)
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBcheck] [discriminator] Plan 2: Capturing Reference RTL outputs as ground truth...
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBcheck] [discriminator] Capturing Reference RTL outputs as ground truth...
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBcheck] [discriminator] Reference RTL check results: failed scenarios = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10], passed scenarios = []
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBcheck] [discriminator] Captured Reference RTL outputs for 10 scenarios
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBcheck] [discriminator] Plan 2: Captured outputs for 10 scenarios
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBcheck] [discriminator] Discriminating the testbench, NO.0 discrimination
2026-04-22 00:00:54 | POSITIVE | [spi_controller_simple] [TBcheck] [discriminator] TB_discriminating finished, TB passed, wrong scenarios: [], scenario pass ratio: 0/10
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBcheck] Testbench passed the funccheck
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [TBcheck] self funccheck finished. Next Action: [pass]
2026-04-22 00:00:54 | INFO | [spi_controller_simple]
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [spi_controller_simple] Starting Coverage-Guided Agent (CGA)...
2026-04-22 00:00:54 | INFO | [spi_controller_simple] [spi_controller_simple] Running Semantic Analysis (Layer 0)...
2026-04-22 00:00:54 | INFO | [spi_controller_simple] FSM detected: state (4 states)
2026-04-22 00:00:54 | INFO | [spi_controller_simple] Total function points identified: 8
2026-04-22 00:00:54 | INFO | [spi_controller_simple] Energy allocator initialized: 8 targets
2026-04-22 00:00:54 | INFO | [spi_controller_simple] Diversity injector initialized with history file: saves/0420~0426/MyExperimentsSimple/SimpleSPI_20260421_214748/spi_controller_simple/CGA/test_history.json
2026-04-22 00:00:54 | INFO | [spi_controller_simple] Quality evaluator initialized
2026-04-22 00:00:54 | INFO | [spi_controller_simple] --- CGA Iter 0 (Baseline) ---
2026-04-22 00:00:56 | INFO | [spi_controller_simple] Baseline Coverage: 77.55%
2026-04-22 00:00:56 | INFO | [spi_controller_simple] --- CGA Iter 1 / 15 ---
2026-04-22 00:00:56 | INFO | [spi_controller_simple] Target: Counter_bit_count
2026-04-22 00:00:56 | INFO | [spi_controller_simple] Asking LLM to fix missing logic (Current: 77.55%)...
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Quality Evaluation: diversity=1.00
2026-04-22 00:01:27 | SUCCESS | [spi_controller_simple] Coverage Improved! +16.33% (77.55% -> 93.88%)
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Unreachable analysis: 1 truly unreachable, 2 potentially coverable
2026-04-22 00:01:27 | WARNING | [spi_controller_simple] Coverage reached target, but found 1 unreachable branches: | location: autoline/TB_cga.py, func: run, line: 1651 | caller: location: autoline/TB_autoline.py, func: run_TBCGA, line: 302
2026-04-22 00:01:27 | WARNING | [spi_controller_simple] Line 93: default: begin - default branch in fully-covered case statement | location: autoline/TB_cga.py, func: run, line: 1653 | caller: location: autoline/TB_autoline.py, func: run_TBCGA, line: 302
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Stopping: unreachable branches exist by RTL design
2026-04-22 00:01:27 | INFO | [spi_controller_simple] CGA Finished. Final Coverage: 93.88%
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Unreachable analysis: 1 truly unreachable, 2 potentially coverable
2026-04-22 00:01:27 | WARNING | [spi_controller_simple] Unreachable branches found: 1 lines | location: autoline/TB_cga.py, func: run, line: 1714 | caller: location: autoline/TB_autoline.py, func: run_TBCGA, line: 302
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Unreachable branches report saved to: saves/0420~0426/MyExperimentsSimple/SimpleSPI_20260421_214748/spi_controller_simple/CGA/unreachable_branches_report.txt
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Energy report saved to saves/0420~0426/MyExperimentsSimple/SimpleSPI_20260421_214748/spi_controller_simple/CGA/energy_report.txt
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Diversity report saved to saves/0420~0426/MyExperimentsSimple/SimpleSPI_20260421_214748/spi_controller_simple/CGA/diversity_report.txt
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Quality evaluation report saved to saves/0420~0426/MyExperimentsSimple/SimpleSPI_20260421_214748/spi_controller_simple/CGA/quality_evaluation_report.txt
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Semantic Coverage: 90.37%
2026-04-22 00:01:27 | INFO | [spi_controller_simple] Saved optimized TB to: saves/0420~0426/MyExperimentsSimple/SimpleSPI_20260421_214748/spi_controller_simple/final_TB.v
2026-04-22 00:01:27 | INFO | [spi_controller_simple] [TBeval] Eval 1: Golden RTL checking begins
2026-04-22 00:01:27 | FAILED | [spi_controller_simple] [TBeval] Eval 1: Golden RTL checking failed!
2026-04-22 00:01:27 | INFO | [spi_controller_simple] [TBeval] [spi_controller_simple] Eval 2/2b is skipped because Eval 1 failed
2026-04-22 00:01:27 | INFO | [spi_controller_simple]
2026-04-22 00:01:27 | INFO |
########## Analyze of Chatbench_RunInfo ##########
#### pass numbers:
Eval2 : 0
Eval1 : 0
Eval0 : 1
total : 1 (Failed: 0)
passed TB by autoline reboot action (from TB3_check): 0
passed TB by functional corrector: 0
#### CGA Coverage Info:
Average Coverage : 93.88%
Max Coverage : 93.88%
Min Coverage : 93.88%
#### tokens and cost:
average prompt tokens: 24557
average completion tokens: 20046
total cost: 1.6939
average cost: 1.6939
#### time:
average time: 8018.63s
#### debug info table:
FUNCTIONAL debug info table:
(debugged here means functional debugging)
| un-func-debugged | func-debugged | total |
failed | 0 | 0 | 0 |
Eval0 | 0 | 1 | 1 |
Eval1 | 0 | 0 | 0 |
Eval2 | 0 | 0 | 0 |
#### Eval2 ratio:
#### CGA Coverage Detail List:
Task ID | Coverage
----------------------------------------
spi_controller_simple | 93.88%
loose Eval2 pass metric applied: 0.8
Reference in New Issue Copy Permalink