Detailed Data

Test	bdd								bmc								clp								hassel								hassel_diff								pdr
	0	1	2	3	4	5	6	7	0	1	2	3	4	5	6	7	0	1	2	3	4	5	6	7	0	1	2	3	4	5	6	7	0	1	2	3	4	5	6	7	0	1	2	3	4	5	6	7
cloud/base/01.sat.smt2	0.22	0.21	0.22	0.21	2.02	0.39	2.01	0.39	0.32	0.32	0.32	0.32	0.70	0.71	0.70	0.70	0.27	0.44	0.27	0.27	0.70	0.70	0.70	0.69	0.59	0.66	0.60	0.66	8.84	8.62	8.84	8.61	0.21	0.21	0.21	0.21	T/O	T/O	T/O	T/O	0.29	0.30	0.30	0.29	0.71	0.72	0.70	0.71
cloud/base/02.unsat.smt2	0.22	0.40	0.22	0.39	2.15	0.42	2.15	0.42	T/O	T/O	T/O	T/O	0.71	0.71	0.71	0.71	0.41	0.41	0.40	0.41	0.71	0.71	0.71	0.71	0.58	0.85	0.59	0.86	9.50	9.35	9.53	9.37	0.22	0.20	0.22	0.20	T/O	T/O	T/O	T/O	0.54	0.53	0.53	0.54	0.73	0.72	0.72	0.72
cloud/base/03.sat.smt2	0.22	0.21	0.21	0.21	1.92	0.31	1.92	0.31	0.31	0.30	0.30	0.30	0.70	0.69	0.69	0.69	0.27	0.27	0.26	0.26	0.68	0.69	0.68	0.68	0.27	0.26	0.27	0.26	3.33	1.86	3.33	1.84	0.21	0.20	0.21	0.20	268.89	126.06	273.36	127.04	0.28	0.29	0.29	0.28	0.69	0.69	0.70	0.69
cloud/base/04.unsat.smt2	0.22	0.35	0.22	0.36	2.09	0.39	2.09	0.40	T/O	T/O	T/O	T/O	0.71	0.70	0.70	0.70	0.35	0.36	0.35	0.34	0.69	0.69	0.71	0.69	0.54	0.87	0.53	0.86	9.02	8.79	9.03	8.82	0.22	0.21	0.21	0.20	T/O	T/O	T/O	T/O	0.50	0.50	0.50	0.50	0.72	0.71	0.70	0.71
cloud/base/05.sat.smt2	0.22	0.20	0.22	0.21	2.44	0.37	2.42	0.37	0.33	0.33	0.33	0.34	0.69	0.69	0.69	0.70	0.27	0.27	0.27	0.27	0.69	0.68	0.68	0.69	0.21	0.20	0.22	0.20	2.31	0.31	2.30	0.31	0.21	0.20	0.21	0.20	2.32	0.30	2.31	0.30	0.35	0.34	0.34	0.35	0.69	0.70	0.71	0.70
cloud/base/06.unsat.smt2	0.22	0.21	0.22	0.21	2.46	0.37	2.42	0.37	T/O	T/O	T/O	T/O	0.70	0.70	0.69	0.69	0.27	0.28	0.27	0.28	0.69	0.69	0.69	0.70	0.22	0.20	0.22	0.21	2.31	0.31	2.31	0.31	0.22	0.20	0.22	0.20	2.31	0.30	2.33	0.31	0.39	0.40	0.40	0.40	0.70	0.70	0.71	0.70
cloud/more_nodes/01.sat.smt2	2.82	2.28	2.80	2.28	89.55	5.56	90.60	5.56	280.08	283.70	275.53	287.75	24.45	24.26	23.59	24.66	??	??	??	??	??	??	??	??	5.56	4.89	5.48	4.87	158.98	85.37	161.09	85.26	2.80	2.21	2.80	2.21	T/O	T/O	T/O	T/O	3.62	3.59	3.58	3.58	9.11	8.88	12.08	9.03
cloud/more_nodes/02.unsat.smt2	2.84	2.24	2.83	2.22	87.22	5.97	87.29	5.48	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	??	??	??	??	5.47	4.81	5.58	4.82	156.09	84.31	157.48	84.37	2.80	2.20	2.79	3.22	T/O	T/O	T/O	T/O	25.92	26.04	25.62	26.13	16.26	17.20	15.76	16.09
cloud/more_nodes/03.sat.smt2	3.16	2.23	3.16	2.23	130.27	5.87	124.41	5.87	T/O	T/O	T/O	T/O	34.17	34.24	34.02	34.47	??	??	??	??	??	??	??	??	3.24	2.30	3.21	2.29	124.42	5.28	123.67	5.28	3.16	2.19	3.13	2.19	129.86	6.05	129.97	5.97	14.05	13.96	14.13	14.01	12.86	12.93	13.00	13.11
cloud/more_nodes/04.unsat.smt2	3.16	2.23	3.17	2.22	130.10	5.91	128.54	6.08	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	??	??	??	??	3.30	2.36	3.33	2.35	127.59	5.29	129.16	5.35	3.21	2.23	3.20	2.27	126.40	6.07	126.70	6.09	33.87	34.38	33.39	33.66	18.60	19.12	17.98	18.71
cloud/more_nodes/05.sat.smt2	4.51	2.26	2.70	2.24	87.26	4.73	85.11	4.93	236.97	237.11	237.01	241.84	23.39	23.35	23.90	23.37	??	??	??	??	??	??	??	??	2.93	2.45	2.95	2.42	94.76	10.45	93.72	10.49	2.68	2.21	2.67	2.19	T/O	T/O	T/O	T/O	3.41	3.43	3.43	3.42	8.91	8.92	13.63	9.06
cloud/more_nodes/06.unsat.smt2	2.15	2.17	2.18	2.18	5.09	4.68	5.09	4.75	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O	??	??	??	??	??	??	??	??	2.16	2.16	2.20	2.17	4.96	4.52	5.08	4.55	2.18	2.16	2.16	2.16	4.95	4.55	4.95	4.55	13.89	12.53	12.46	12.45	13.72	14.09	13.91	16.67
cloud/more_nodes_long/01.sat.smt2	37.04	13.12	37.23	13.13	T/O	143.62	T/O	143.60	T/O	T/O	T/O	T/O	M/O	M/O	M/O	M/O	??	??	??	??	??	??	??	??	150.77	98.26	123.13	86.93	T/O	T/O	T/O	T/O	38.57	12.75	38.29	12.71	T/O	T/O	T/O	T/O	35.13	36.04	34.80	36.53	95.25	93.06	96.89	94.88
cloud/more_nodes_long/02.unsat.smt2	39.80	12.98	39.74	12.97	T/O	140.63	T/O	173.75	T/O	T/O	T/O	T/O	M/O	M/O	M/O	M/O	??	??	??	??	??	??	??	??	141.52	87.69	145.16	87.47	T/O	T/O	T/O	T/O	41.29	12.92	41.22	12.90	T/O	T/O	T/O	T/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O
cloud/more_nodes_long/03.sat.smt2	58.47	15.99	56.71	13.28	T/O	140.40	T/O	140.57	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O	??	??	??	??	??	??	??	??	57.65	15.57	57.35	15.49	T/O	123.63	T/O	124.09	56.94	12.95	56.75	12.87	T/O	129.99	T/O	131.43	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O
cloud/more_nodes_long/04.unsat.smt2	55.93	13.31	55.46	13.26	T/O	140.12	T/O	141.54	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O	??	??	??	??	??	??	??	??	57.10	15.33	56.70	15.38	T/O	124.40	T/O	123.75	55.62	12.82	55.26	12.74	T/O	131.71	T/O	130.90	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O
cloud/more_nodes_long/05.sat.smt2	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	??	??	??	??	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	38.92	12.70	38.42	13.00	T/O	T/O	T/O	T/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O
cloud/more_nodes_long/06.unsat.smt2	12.47	11.93	12.97	11.96	118.26	101.87	118.59	102.38	M/O	M/O	M/O	M/O	T/O	M/O	M/O	M/O	??	??	??	??	??	??	??	??	13.55	11.87	12.44	11.81	119.54	101.18	118.40	100.81	12.42	13.89	12.55	11.89	118.07	102.32	118.90	101.61	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O
cloud/more_services/01.sat.smt2	8.53	23.91	7.68	101.80	25.55	20.26	114.79	16.31	6.45	6.50	6.43	6.44	8.97	9.67	8.93	8.91	??	??	??	??	??	??	??	??	240.70	T/O	67.97	T/O	T/O	T/O	T/O	T/O	6.75	6.06	7.53	9.98	T/O	T/O	T/O	T/O	6.72	6.78	6.77	6.78	9.41	9.31	9.33	9.89
cloud/more_services/02.sat.smt2	8.64	23.96	7.60	102.81	25.56	20.04	115.82	15.89	7.52	7.33	7.17	7.31	10.67	10.65	11.78	10.60	??	??	??	??	??	??	??	??	232.84	T/O	70.43	T/O	T/O	T/O	T/O	T/O	6.33	6.06	7.52	10.01	T/O	T/O	T/O	T/O	8.71	8.59	8.64	8.65	11.03	11.13	10.97	10.95
cloud/more_services/03.unsat.smt2	5.90	5.89	6.70	6.33	7.73	7.69	12.50	11.85	T/O	T/O	T/O	T/O	9.64	9.69	9.61	9.67	??	??	??	??	10.38	9.24	9.17	9.11	5.83	5.87	6.34	6.27	7.45	7.43	12.13	11.60	5.82	5.83	6.31	6.23	7.46	7.42	12.14	11.58	8.54	8.59	8.57	8.59	10.03	10.08	10.06	10.08
cloud/more_services/04.sat.smt2	10.38	10.70	7.60	6.24	14.24	7.26	124.92	11.82	7.13	7.13	7.27	7.13	68.42	68.20	68.91	69.35	??	??	??	??	??	??	??	??	5.83	7.91	7.56	6.13	14.29	7.02	121.98	10.63	5.87	5.81	7.54	6.15	13.95	7.04	122.59	10.52	8.73	8.69	8.64	8.71	72.19	70.41	69.19	70.79
cloud/more_services/05.unsat.smt2	11.42	23.38	8.80	131.41	25.06	18.20	136.94	17.79	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	10.87	11.41	10.97	13.60	249.37	T/O	82.02	T/O	T/O	T/O	T/O	T/O	5.96	6.15	8.12	10.91	T/O	T/O	T/O	T/O	107.87	121.23	104.54	132.59	11.95	11.93	12.07	11.90
cloud/more_services/06.unsat.smt2	15.76	15.84	13.60	12.16	21.03	13.40	124.83	17.72	T/O	T/O	T/O	T/O	9.30	9.35	9.34	9.27	??	??	??	??	8.91	8.90	8.87	8.84	12.38	11.84	13.45	12.23	18.93	12.92	121.88	16.80	11.83	11.76	13.95	12.39	18.79	12.86	123.92	16.62	128.27	124.80	123.93	137.84	9.74	9.75	9.69	9.87
cloud/most_nodes/01.sat.smt2	112.88	58.62	112.49	59.06	T/O	231.23	T/O	230.60	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	??	??	??	??	133.08	81.24	137.78	80.86	T/O	T/O	T/O	T/O	112.73	58.43	117.16	59.00	T/O	T/O	T/O	T/O	89.39	86.78	87.97	86.88	171.67	173.45	170.45	171.39
cloud/most_nodes/02.unsat.smt2	112.84	58.75	112.52	58.69	T/O	233.60	T/O	230.36	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	??	??	??	??	136.95	79.80	135.75	78.90	T/O	T/O	T/O	T/O	112.15	58.38	113.02	58.50	T/O	T/O	T/O	T/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O
cloud/most_nodes/03.sat.smt2	146.54	60.21	145.07	62.15	T/O	230.69	T/O	229.59	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	??	??	??	??	147.48	59.19	146.87	59.66	T/O	224.35	T/O	224.87	146.77	58.16	147.23	58.39	T/O	238.06	T/O	238.64	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O
cloud/most_nodes/04.unsat.smt2	144.10	58.61	144.14	59.00	T/O	232.12	T/O	231.07	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	??	??	??	??	148.23	59.35	147.03	59.60	T/O	225.89	T/O	227.32	148.14	58.58	146.57	58.33	T/O	240.23	T/O	240.03	M/O	M/O	M/O	M/O	M/O	M/O	M/O	M/O
stanford/01.sat.smt2	T/O	T/O	T/O	11.78	T/O	T/O	T/O	T/O	8.37	9.97	8.51	8.43	27.81	28.59	27.84	27.80	??	??	??	??	??	??	??	??	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	11.13	5.78	11.75	5.85	T/O	T/O	T/O	T/O	3.98	3.95	3.97	3.97	21.27	22.27	22.21	21.05
stanford/02.unsat.smt2	T/O	T/O	T/O	16.50	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	??	??	??	??	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	10.49	5.26	11.20	5.43	T/O	T/O	T/O	T/O	4.39	4.40	4.40	4.40	21.50	20.99	24.24	21.02
stanford/03.sat.smt2	T/O	T/O	T/O	13.24	T/O	T/O	T/O	T/O	62.46	65.02	56.68	55.49	40.76	40.58	40.74	40.92	??	??	??	??	??	??	??	??	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	14.35	6.25	13.79	6.31	T/O	T/O	T/O	T/O	26.88	26.67	26.84	26.70	23.65	23.46	24.57	23.66
stanford/04.unsat.smt2	T/O	101.13	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	??	??	??	??	??	??	??	??	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	8.67	2.38	8.15	2.40	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	23.33	23.21	25.95	23.08
stanford/loop01.sat.smt2	T/O	T/O	T/O	11.42	T/O	T/O	T/O	T/O	23.17	21.70	23.01	20.77	31.10	31.18	31.20	31.16	??	??	??	??	??	??	??	??	T/O	T/O	T/O	T/O	T/O	T/O	T/O	T/O	13.03	4.12	13.38	4.27	T/O	T/O	T/O	T/O	11.63	11.72	15.41	11.79	21.91	22.12	21.98	21.92
Total	2,600.7s	2,023.3s	2,586.5s	1,332.2s	4,980.0s	3,516.1s	5,482.4s	3,549.8s	7,233.1s	7,239.4s	7,222.6s	7,236.1s	5,092.9s	5,094.0s	5,094.0s	5,094.4s	8,101.8s	8,102.0s	8,101.8s	8,101.8s	7,234.3s	7,233.7s	7,233.2s	7,235.7s	3,558.4s	3,255.9s	3,035.2s	3,242.7s	5,962.3s	4,951.3s	6,179.9s	4,964.4s	881.9s	391.5s	893.7s	405.7s	7,593.0s	6,713.0s	7,817.2s	6,725.6s	3,237.4s	3,244.5s	3,229.5s	3,269.0s	2,986.6s	2,986.6s	2,998.2s	2,987.4s

Best Configs per Tool

Tool	Config	Total Time
bdd	3	1,332.2s
bmc	4	5,092.9s
clp	6	7,233.2s
hassel	2	3,035.2s
hassel_diff	1	391.5s
pdr	5	2,986.6s

Statistics

Status	bdd	bmc	clp	hassel	hassel_diff	pdr
OK	205	112	60	174	180	196
M/O	0	39	0	0	0	64
T/O	59	113	0	90	84	4
UNKNOWN	0	0	204	0	0	0

Options List

0: ''
1: 'fixedpoint.unbound_compressor=false'
2: 'fixedpoint.similarity_compressor=false'
3: 'fixedpoint.similarity_compressor=false fixedpoint.unbound_compressor=false'
4: 'fixedpoint.bit_blast=true'
5: 'fixedpoint.bit_blast=true fixedpoint.unbound_compressor=false'
6: 'fixedpoint.bit_blast=true fixedpoint.similarity_compressor=false'
7: 'fixedpoint.bit_blast=true fixedpoint.similarity_compressor=false fixedpoint.unbound_compressor=false'