From hogsett at csl.sri.com Fri Jan 13 11:51:09 2006 From: hogsett at csl.sri.com (Michael Hogsett) Date: Fri Jan 13 11:32:18 2006 Subject: [Smtcomp] test Message-ID: <43C804AD.3020903@csl.sri.com> test From demoura at csl.sri.com Fri Jan 13 11:57:46 2006 From: demoura at csl.sri.com (Leonardo de Moura) Date: Fri Jan 13 11:38:20 2006 Subject: [SMTCOMP] teste Message-ID: <16AEBA36-2CB1-49A5-A9C8-A3513503B796@csl.sri.com> From demoura at csl.sri.com Fri Jan 13 11:58:59 2006 From: demoura at csl.sri.com (Leonardo de Moura) Date: Fri Jan 13 11:39:37 2006 Subject: [SMTCOMP] Testing Message-ID: <86EF8C74-AC6C-4BFD-9287-0ADC07356BA3@csl.sri.com> Simple test From stump at cse.wustl.edu Wed Jan 25 08:21:06 2006 From: stump at cse.wustl.edu (Aaron Stump) Date: Wed Jan 25 08:00:02 2006 Subject: [SMTCOMP] request for comment Message-ID: <200601251621.k0PGL6805641@priam.cse.wustl.edu> Dear SMT-COMP interest group, A month ago or so we sent out our current working draft of the SMT-COMP rules for 2006 for your comment. We have received very little feedback as of yet, and we would really appreciate your input. So if you haven't already done so, please take a moment to look at the draft and let us know any changes you think should be made. We will prepare the final version of the rules soon, so please give us your comments no later than February 1. The draft rules are attached below. Best wishes, Aaron Stump (writing also for Clark Barrett and Leonardo de Moura) -------------- next part -------------- A non-text attachment was scrubbed... Name: smtcomp06rules.pdf Type: application/pdf Size: 41068 bytes Desc: not available Url : http://lists.csl.sri.com/pipermail/smtcomp/attachments/20060125/b8142412/smtcomp06rules.pdf From hsheini at eecs.umich.edu Tue Jan 31 15:58:19 2006 From: hsheini at eecs.umich.edu (Hossein Sheini) Date: Tue Jan 31 15:36:23 2006 Subject: [SMTCOMP] request for comment Message-ID: <1138751899.43dff99bd5155@mail.eecs.umich.edu> Dear Aaron, First I would like to thank all of you for the very successful competition you held in 2005. I participated in SMT-COMP'05 with my solver, Ario. I just have a few comments/suggestions for the SMT-COMP'06, as follows: 1. I think it is very helpful to have your method for selecting benchmarks from the benchmark pool as described in your "Benchmark Selection" section. However, I do have a question on how the organizers are going to determine the difficulty of each benchmark considering that different solvers perform differently on those benchmarks. My suggestion is to use the average data as collected in SMT-COMP'05 to regulate that decision. 2. There were instances in SMT-COMP'05 that were added at the last minute, not included in the released final version of benchmarks suites, i.e. check/int_incompleteness2.smt. I just wanted to suggest that if the same could happen this time, it is better to be mentioned in the documents that all solvers are required to be able to read the SMT header format (like note, comment, etc sections) correctly. 3. In SMT-COMP'05, there were a few solvers that performed buggy in many instances. My suggestion is to set up a rule that if a solver passed a threshold in the number of bugs, it should be eliminated from the competition because I dont know how its performance on those instances that it had no bugs can be trusted. (example of such solver is the performance of HTP in QF_LRA category considering its buggy performance in other categories) 4. I am very much excited about the short presentation as suggested for SMT-COMP'06. I think it is a very good idea and gives everyone an opportunity to get to know the spec of other solvers after seeing their performance. I do appreciate your effort and would like to say that it played an important role for me to get to know the weaknesses and advantages of my solver better and to focus my work accordingly. I also would like to express my interest to get more involved in the SMT community and help by any means in any of these activities. with best wishes, Hossein Sheini EECS PhD Candidate University of Michigan at Ann Arbor www.eecs.umich.edu/~hsheini Quoting Aaron Stump : > Dear SMT-COMP interest group, > > A month ago or so we sent out our current working draft of the SMT-COMP > rules for 2006 for your comment. We have received very little feedback > as of yet, and we would really appreciate your input. So if you haven't > already done so, please take a moment to look at the draft and let us > know any changes you think should be made. We will prepare the final > version of the rules soon, so please give us your comments no later than > February 1. The draft rules are attached below. > > Best wishes, > Aaron Stump > (writing also for Clark Barrett and Leonardo de Moura) > >