Files
TBgen_App/data/__pycache__/probset.cpython-312.pyc

88 lines
14 KiB
Plaintext
Raw Normal View History

2026-03-30 16:46:48 +08:00
<EFBFBD>
P<>Yiy*<00><00><><00>dZedk(rddlZejj d<04>ddlmZddlZddl m
Z
d<07>Z Gd<08>d <09>Z Gd
<EFBFBD>d <0B>Z Gd <0C>d e <0C>Zedk(re <0B>yy)z<>
Description : the class to manage the problem set data
Author : Ruidi Qiu (r.qiu@tum.de)
Time : 2024/3/6 14:01:22
LastEdited : 2024/8/13 00:56:57
<EFBFBD>__main__<5F>N<>.)<01>deepcopy)<01>Configc<00><><00>d}d}t||dg<01><04>}t|j<00>t|jd<00>y)Nzdata/HDLBits/HDLBits_data.jsonlz'data/HDLBits/HDLBits_circuit_type.jsonl<6E>rule110)<02>circuit_type_path<74>
only_tasksr)<04>HDLBitsProbset<65>print<6E>num<75>data)<03>HDLBITS_DATA_PATH<54>CIRCUIT_TYPE_PATH<54>probsets <20>2/home/zhang/CorrectBench/TBgen_App/data/probset.py<70>mainrs><00><00>9<><15>A<><15><1C>.<2E>BS<42>aj<61>`k<>l<>G<EFBFBD> <09>'<27>+<2B>+<2B><16> <09>'<27>,<2C>,<2C>q<EFBFBD>/<2F><1A>c <00>v<00>eZdZdZdgdgifdedededededef d <09>Zed
<EFBFBD><00>Z dgifd <0B>Z
d <0C>Z d <0A>Z dd<0E>Z dd<0F>Zy)<11>dictlistz<74>
- a class to manage the list of dict
- form:
{
id_key: "xxx", #the key to identify the dict
content1: xxx,
content2: xxx,
...
}
N<>id_key<65>path<74>moreinfo_path_list<73>only<6C>exclude<64>filterc<00><><00>||_|<02><> tj|<02>|_|gk7r; |D<00>cgc]}tj|<07><00><02>}}|D]} |j | <09><00>|j |<06>|j|d<02><03>|j|<05>yg|_y#tj|<02>|_Y<00><>xYwcc}w#|D<00>cgc]}tj|<07><00><02>ncc}w}}Y<00><>xYw)z9
if path is None, will not load any data
NF<4E><01> del_by_list)r<00>ls<6C>load_json_linesr<00>load_json_dict<63>merger<00> del_items)
<EFBFBD>selfrrrrrr<00> moreinfo_path<74>moreinfo<66>infos
r<00>__init__zdictlist.__init__#s<><00><00><1D><04> <0B> <0F> <1B> 4<><1E>.<2E>.<2E>t<EFBFBD>4<><04> <09>"<22>R<EFBFBD>'<27>j<01>Wi<57>j<>m<EFBFBD><02> 2<> 2<>=<3D> A<>j<>H<EFBFBD>j<>%<25>%<25>D<EFBFBD><18>J<EFBFBD>J<EFBFBD>t<EFBFBD>$<24>%<25> <10>K<EFBFBD>K<EFBFBD><06> <1F> <10>N<EFBFBD>N<EFBFBD>4<EFBFBD>U<EFBFBD>N<EFBFBD> 3<> <10>N<EFBFBD>N<EFBFBD>7<EFBFBD> #<23><1A>D<EFBFBD>I<EFBFBD><49> 4<><1E>-<2D>-<2D>d<EFBFBD>3<><04> <09><> k<01><>j<01>Vh<56>i<>]<5D><02> 1<> 1<>-<2D> @<40>i<><69>i<>H<EFBFBD>i<>s3<00>B#<00>C <00>C<04> C <00>#C<03>C <00> C5<03>C,<06>+C5c<00>,<00>t|j<00>S<00>N)<02>lenr<00>r%s rr z dictlist.num:s<00><00><12>4<EFBFBD>9<EFBFBD>9<EFBFBD>~<7E>rc<00>n<00>|j|d<01><02>|j|<02>|j|<03>y)NFr)r$r)r%rrrs r<00>
data_cleanzdictlist.data_clean>s*<00><00> <0C><0E><0E>t<EFBFBD><15><0E>/<2F> <0C><0E><0E>w<EFBFBD><1F> <0C> <0B> <0B>F<EFBFBD>rc<00><><00>|jD]}||j|k(s<01>|cStd|jz|zdz<00><00>)Nz,Cannot find the problem infomation with %s: r)rr<00>
ValueError)r%<00>id<69> prob_datas r<00>find_data_by_idzdictlist.find_data_by_idCsO<00><00><1D><19><19> !<21>I<EFBFBD><18><14><1B><1B>%<25><12>+<2B> <20> <20> !<21><19>G<><14><1B><1B>U<>XZ<58>Z<>]`<60>`<60>a<>arc<00><><00>|jD]V}|D]O}||j||jk(s<01>#|j<00>D]\}}||jk7s<01>|||<<00><00>Q<00>Xy<01>z,merge additional data into the original dataN)rr<00>items)r%<00>additional_datar<00>add_data<74>key<65>values rr#zdictlist.mergeIsm<00><00><18>I<EFBFBD>I<EFBFBD> .<2E>D<EFBFBD>+<2B> .<2E><08><17><04> <0B> <0B>$<24><08><14><1B><1B>(=<3D>=<3D>&.<2E>n<EFBFBD>n<EFBFBD>&6<>.<2E>
<EFBFBD><03>U<EFBFBD><1E>$<24>+<2B>+<2B>-<2D>(-<2D>D<EFBFBD><13>I<EFBFBD>.<2E> .<2E> .rc<00>Z<00>|rM|j<00>D]9\}}|jD<00>cgc]}|j|<03>|k(s<01>|<05><02>c}|_<00>;yt|j<00>}|j<00>D])\}}|D<00>cgc]}|j|<03>|k(s<01>|<05><02>}}<05>+|Scc}wcc}w<00>a!
#### Function
- filtering the data by the key and value.
- only the data that has the key and value will remain
- the output will always be the filtered data, but I recommend to directly use `self.data` to get the filtered data if del_en is True
#### Input
- filter_dict: dict; the key and value to filter the data
- del_en: bool; if True, the data that doesn't have the key and value will be deleted from the data. If False, the data will not change but output the filtered data
N)r7r<00>getr)r%<00> filter_dict<63>del_enr:r;r3<00> filtered_datas rrzdictlist.filterRs<><00><00> <12>)<29>/<2F>/<2F>1<> b<01>
<EFBFBD><03>U<EFBFBD>8<<3C> <09> <09>a<>9<EFBFBD>Y<EFBFBD>]<5D>]<5D>SV<53>EW<45>[`<60>E`<60>Y<EFBFBD>a<><04> <09> b<01>%<25>T<EFBFBD>Y<EFBFBD>Y<EFBFBD>/<2F>M<EFBFBD>)<29>/<2F>/<2F>1<> j<01>
<EFBFBD><03>U<EFBFBD><I<> i<>y<EFBFBD>Y<EFBFBD>]<5D>]<5D>[^<5E>M_<4D>ch<63>Mh<4D><19> i<> <0A> i<> j<01> <20> <20><> b<01><>!js<00>B#<06>B#<06>?B(<06>B(c<00><><00>|<01>l|gk7rf|r2|jD<00>cgc]}||j|vs<01>|<03><02>c}|_y|jD<00>cgc]}||j|vs<01>|<03><02>c}|_yyycc}wcc}w<00>z<>
- id_list: list of ids
- del_by_list: bool; if True, data having the task_id in the list will be deleted. If False, the data that doesn't have the task_id in the list will be deleted
N)rr)r%<00>id_listrr3s rr$zdictlist.del_itemseso<00><00> <13> <1E>7<EFBFBD>b<EFBFBD>=<3D><1A>8<<3C> <09> <09>k<>9<EFBFBD>Y<EFBFBD>t<EFBFBD>{<7B>{<7B>E[<5B>cj<63>Ej<45>Y<EFBFBD>k<><04> <09>8<<3C> <09> <09>g<>9<EFBFBD>Y<EFBFBD>t<EFBFBD>{<7B>{<7B>E[<5B>_f<5F>Ef<45>Y<EFBFBD>g<><04> <09> $1<> <1E><>k<><6B>gs<00>A0<04>A0<04>
A5<04>!A5<04>T)<0F>__name__<5F>
__module__<EFBFBD> __qualname__<5F>__doc__<5F>str<74>list<73>dictr)<00>propertyr r/r4r#rr$<00>rrrrs<><00><00>
<08>-1<>"<22>X\<5C>km<6B>{}<7D><1B>c<EFBFBD><1B><03><1B>T<EFBFBD><1B>SW<53><1B>fj<66><1B>vz<76><1B>.<0E><1E><0E><1E>#<23>B<EFBFBD>r<EFBFBD><1C>
b<01> .<2E>!<21>&
hrrc <00><><00>eZdZdZdgdgifdedededededef d <09>Zgdgifd
<EFBFBD>Zgdgifd <0B>Z dgifd <0C>Z
d <0A>Z dd<0E>Z dd<0F>Z defd<11>Zdefd<13>Zdefd<14>Zed<15><00>Zed<16><00>Zed<17><00>Zy)<19> muti_dictlistz{
- mutiple dictlists, can perform the same operation on all the dictlists
- self.dictlists: a list of dictlist
Nr<00> path_listrrrrc<00>R<00>g|_||_|<02>|j||||||<06>yy)zi
you can only determing the id_key (mostly, "task_id"); the dictlists can be added later
N)<03> dictlistsr<00>load_dictlists)r%rrQrrrrs rr)zmuti_dictlist.__init__vs7<00><00><1C><04><0E><1C><04> <0B> <14> <20> <10> <1F> <1F><06> <09>3E<33>t<EFBFBD>W<EFBFBD>V\<5C> ]<5D> !rc <00>T<00>|D<00>cgc]}t||||||<06><00><02>c}|_ycc}wr+)rrS)r%rrQrrrrrs rrTzmuti_dictlist.load_dictlistss)<00><00>hq<68>r<>`d<>(<28>6<EFBFBD>4<EFBFBD>1C<31>T<EFBFBD>7<EFBFBD>TZ<54>[<5B>r<><04><0E><>rs<00>%c
<00>V<00>|jjt||||||<06><00>yr+)rS<00>appendr)r%rrrrrrs r<00> load_dictlistzmuti_dictlist.load_dictlist<73>s%<00><00> <0C><0E><0E><1D><1D>h<EFBFBD>v<EFBFBD>t<EFBFBD>5G<35><14>w<EFBFBD>X^<5E>_<>`rc<00>L<00>|jD]}|j|||<03><00>yr+)rSr/)r%rrrrs rr/zmuti_dictlist.data_clean<61>s'<00><00><1C><0E><0E> 7<>H<EFBFBD> <14> <1F> <1F><04>g<EFBFBD>v<EFBFBD> 6<> 7rc<00>H<00>|jD]}|j|<01><00>yr6)rSr#)r%r8rs rr#zmuti_dictlist.merge<67>s!<00><00><1C><0E><0E> ,<2C>H<EFBFBD> <14>N<EFBFBD>N<EFBFBD>?<3F> +<2B> ,rc<00>J<00>|jD]}|j||<02><00>yr=)rSr)r%r?r@rs rrzmuti_dictlist.filter<65>s%<00><00><1D><0E><0E> 1<>H<EFBFBD> <14>O<EFBFBD>O<EFBFBD>K<EFBFBD><16> 0<> 1rc<00>J<00>|jD]}|j||<02><00>yrC)rSr$)r%rDrrs rr$zmuti_dictlist.del_items<6D>s'<00><00>
<1D><0E><0E> 5<>H<EFBFBD> <14> <1E> <1E>w<EFBFBD> <0B> 4<> 5r<00>funcc<00><><00>g}|jD]$} |jt||<01>|i|<03><01><01><00>&|S#t$rt d|z<00>Y<00>BwxYw)z<>
this function will perform the function `func` on all the dictlists
for example, if you want to delete the data with task_id in the list, you can use `do("del_items", task_id_list)`
z$The function '%s' is not in dictlist)rSrW<00>getattr<74>AttributeErrorr )r%r]<00>args<67>kwargs<67> result_listrs r<00>dozmuti_dictlist.do<64>so<00><00>
<19> <0B><1C><0E><0E> C<01>H<EFBFBD> C<01><1B>"<22>"<22>#:<3A>7<EFBFBD>8<EFBFBD>T<EFBFBD>#:<3A>D<EFBFBD>#K<>F<EFBFBD>#K<>L<> C<01>
<1B><1A><>"<22> C<01><15><<3C>T<EFBFBD>A<>B<> C<01>s<00>!8<02>A<05>A<05>attrc<00>T<00>|jD<00>cgc]}t||<01><00><02>c}Scc}w)zB
return the list of the attribute of the dictlist
)rSr_)r%rers r<00>accesszmuti_dictlist.access<73>s#<00><00>9=<3D><0E><0E>G<>H<EFBFBD><07><08>$<24>'<27>G<>G<><47>G<><00>%c<00>N<00><02>|j|<01><00>t<00>fd<01><08>D<00><00>S)zO
return True if all the attribute of the dictlist are the same
c3<00>.<00>K<00>|] }|<01>dk(<00><01><00>y<01>w)rNrN)<03>.0re<00> attr_lists <20>r<00> <genexpr>z*muti_dictlist.all_equal.<locals>.<genexpr><3E>s<00><><00><><00>><3E>D<EFBFBD>4<EFBFBD>9<EFBFBD>Q<EFBFBD><<3C>'<27>><3E>s<00>)rg<00>all)r%rerls @r<00> all_equalzmuti_dictlist.all_equal<61>s$<00><><00><19>K<EFBFBD>K<EFBFBD><04>%<25> <09><12>><3E>I<EFBFBD>><3E>><3E>>rc<00>T<00>|jD<00>cgc]}|j<00><02>c}Scc}wr+)rSr )r%rs rr zmuti_dictlist.num<75>s<00><00>-1<>^<5E>^<5E><<3C><18><08> <0C> <0C><<3C><<3C><><rhc<00><00>|jSr+<00>rSr-s r<00>datasetszmuti_dictlist.datasets<74><00> <00><00><13>~<7E>~<7E>rc<00><00>|jSr+rrr-s r<00>groupszmuti_dictlist.groups<70>rtrrE)rFrGrHrIrJrKrLr)rTrXr/r#rr$rdrgrorMr rsrvrNrrrPrPqs<00><00><08>37<33>PR<50>^b<>qs<71>BD<02>^<01>c<EFBFBD>^<01>T<EFBFBD>^<01>4<EFBFBD>^<01>Y]<5D>^<01>lp<6C>^<01>}A<02>^<01>DF<01>D<EFBFBD>Z\<5C>eg<65>s<01>>@<01>d<EFBFBD>TV<54>_a<5F>a<01>#<23>B<EFBFBD>r<EFBFBD>7<>,<2C>
1<>5<> <1B>c<EFBFBD> <1B>H<01>#<23>H<01> ?<3F>S<EFBFBD>?<3F><0E>=<3D><0E>=<3D><0E><1E><0E><1E><0E><1E><0E>rrPc <00>r<00><00>eZdZdZdgdgifdededededef
<EFBFBD>fd<08> Zed <09><00>Z d
<EFBFBD>Z
d <0B>Z d e fd <0A>Z d<0E>Z<0E>xZS)r zB has many similarities with HDLBitsData in HDLBits_data_manager.pyNr<00>more_info_pathsr
<00> exclude_tasks<6B>filter_contentc<00>0<00><01>t<00>|<00>d|||||<05><02>y)N<>task_id)rrrrr)<02>superr))r%rrxr
ryrz<00> __class__s <20>rr)zHDLBitsProbset.__init__<5F>s'<00><><00> <0A><07><18><19><14>/<2F>Xb<58>ly<6C>CQ<02><18> Rrc<00>F<00>|jD<00>cgc]}|d<00><02> c}Scc}w)z*
return a list of task_id
r|<00>r)r%<00>is r<00> task_id_listzHDLBitsProbset.task_id_list<73>s <00><00>
'+<2B>i<EFBFBD>i<EFBFBD>0<><11><01>)<29> <0C>0<>0<><30>0s<00> c<00><<00>|D<00>cgc]}d|i<01><02>c}|_ycc}w)zM
return a dictlist with only the task_id in the task_id_list
r|Nr<4E>)r%r<>r<>s r<00>create_empty_set_via_taskidsz+HDLBitsProbset.create_empty_set_via_taskids<64>s<00><00>.:<3A>:<3A><01>i<EFBFBD><11>^<5E>:<3A><04> <09><>:s<00> c<00>X<00>|jD]}|d|k(s<01> |cStd|z<00><00>)zE
return a dict in all the information of the task_id
r|ztask_id %s not found!!!)rr1)r%r|r<>s r<00>access_data_via_taskidz%HDLBitsProbset.access_data_via_taskid<69>s;<00><00><16><19><19> <19>A<EFBFBD><10><19>|<7C>w<EFBFBD>&<26><18><08> <19><19>2<>g<EFBFBD>><3E>?<3F>?r<00>probset_configc<00><00>|j|j|j|j|jdd<02>}|j
<00>|dj |j
<00>|j<00>|dj |j<00>|jdi|<02><01>|j<00><>t|j<00>tk(rNt<00>}tj|j<00>|_|j }|j#|d<05><06>yt|j<00>t$k(rU|jD]E}t<00>}tj|<05>|_|j }|j#|d<05><06><00>Gyyy)z<>
- if use this, please leave path as Empty when initializing
- probset_config = config.autoline.probset
r)rrxr
ryrzNrxTrrN)rrxrrr<00>gptgenRTL_pathrW<00> mutant_pathr)<00> exclude_json<6F>typerJr r r"rr<>r$rK)r%r<><00>parasry<00>exclude_task_id_listr<74>s r<00>load_by_configzHDLBitsProbset.load_by_config<69>se<00><00> #<23>'<27>'<27>-<2D>=<3D>=<3D>(<28>-<2D>-<2D>+<2B>3<>3<>,<2C>3<>3<>A<EFBFBD>6<> 
<EFBFBD><05> <1A> (<28> (<28> 4<> <11>#<23> $<24> +<2B> +<2B>N<EFBFBD>,I<>,I<> J<> <19> %<25> %<25> 1<> <11>#<23> $<24> +<2B> +<2B>N<EFBFBD>,F<>,F<> G<><15><04> <0A> <0A><1E><05><1E> <19> &<26> &<26> 2<><13>N<EFBFBD>/<2F>/<2F>0<>C<EFBFBD>7<> .<2E> 0<> <0A>%'<27>%6<>%6<>~<7E>7R<37>7R<37>%S<> <0A>"<22>'4<>'A<>'A<>$<24><14><0E><0E>3<><14><0E>F<><15>n<EFBFBD>1<>1<>2<>d<EFBFBD>:<3A>$2<>$?<3F>$?<3F>K<01>L<EFBFBD>$2<>$4<>M<EFBFBD>)+<2B>):<3A>):<3A><<3C>)H<>M<EFBFBD>&<26>+8<>+E<>+E<>(<28><18>N<EFBFBD>N<EFBFBD>#7<>T<EFBFBD>N<EFBFBD>J<> K<01>;<3B> 3rc<00>@<00>ddl}|j|j<00>S)z&
return a random task
rN)<03>random<6F>choicer)r%r<>s r<00> random_taskzHDLBitsProbset.random_tasks<00><00> <16><15>}<7D>}<7D>T<EFBFBD>Y<EFBFBD>Y<EFBFBD>'<27>'r)rFrGrHrIrJrKrLr)rMr<>r<>r<>rr<>r<><00> __classcell__)r~s@rr r <00>sy<00><><00>L<> $<24>2<EFBFBD>t<EFBFBD>hj<68>AC<02>R<02>C<EFBFBD>R<02>d<EFBFBD>R<02>$<24>R<02>cg<63>R<02>{<>R<02><0E>1<><0E>1<> ;<3B> @<01>K<01>F<EFBFBD>K<01><(rr )rIrF<00>sysrrW<00>copyr<00> loader_saverr <00>configrrrrPr rNrr<00><module>r<>sr<00><01><04> <0C>z<EFBFBD><19><0E><07>H<EFBFBD>H<EFBFBD>O<EFBFBD>O<EFBFBD>C<EFBFBD><18><19><19><19><1B>Xh<01>Xh<01>tV<1E>V<1E>r?(<28>X<EFBFBD>?(<28>B <0C>z<EFBFBD><19><08>F<EFBFBD>r