From pocmatos at gmail.com Thu Apr 27 08:51:31 2006 From: pocmatos at gmail.com (Paulo J. Matos) Date: Thu Apr 27 08:51:48 2006 Subject: [SMTCOMP] Sudoku based instances Message-ID: <11b141710604270851l16d7270djace5e1a2ef4917db@mail.gmail.com> Hi all, I have generated some sudoku based instances. Would those be interesting for SMTCOMP? Want me to send them in? Cheers, -- Paulo Jorge Matos - pocm at sat inesc-id pt Web: http://sat.inesc-id.pt/~pocm Computer and Software Engineering INESC-ID - SAT Group From Hyondeuk.Kim at colorado.edu Thu Apr 27 10:00:22 2006 From: Hyondeuk.Kim at colorado.edu (Hyondeuk.Kim@colorado.edu) Date: Thu Apr 27 10:00:29 2006 Subject: [SMTCOMP] Sudoku based instances In-Reply-To: <11b141710604270851l16d7270djace5e1a2ef4917db@mail.gmail.com> References: <11b141710604270851l16d7270djace5e1a2ef4917db@mail.gmail.com> Message-ID: <1146157222.4450f8a6afb59@webmail.colorado.edu> Yeah, I think they are interesting. Hyondeuk Quoting "Paulo J. Matos" : > Hi all, > > I have generated some sudoku based instances. Would those be > interesting for SMTCOMP? > Want me to send them in? > > Cheers, > -- > Paulo Jorge Matos - pocm at sat inesc-id pt > Web: http://sat.inesc-id.pt/~pocm > Computer and Software Engineering > INESC-ID - SAT Group > From bruno at csl.sri.com Thu Apr 27 10:57:51 2006 From: bruno at csl.sri.com (Bruno Dutertre) Date: Thu Apr 27 10:57:52 2006 Subject: [SMTCOMP] Sudoku based instances In-Reply-To: <1146157222.4450f8a6afb59@webmail.colorado.edu> References: <11b141710604270851l16d7270djace5e1a2ef4917db@mail.gmail.com> <1146157222.4450f8a6afb59@webmail.colorado.edu> Message-ID: <4451061F.2010704@csl.sri.com> Hyondeuk.Kim@colorado.edu wrote: > Yeah, I think they are interesting. > > Hyondeuk > Well, I'm not so sure these kinds of puzzles are particularly relevant to SMT: - they can be encoded as a big conjunction (no Boolean structure). - they do not require combination of decision procedures - they are probably easy to solve using CSP or pure Boolean SAT techniques and encoding them in arithmetic is likely to be much less efficient. I doubt anybody who really cares about solving Soduku (if such a person exists) would think of using SMT solvers for it. Because of this, I don't think performance of an SMT solver on Sudoku is likely to tell us a lot about performance of the same solver on more realistic applications (e.g., verification problems). We should try to evaluate SMT solvers on problems that require decision procedures, not on problems that can be solved faster/better using other methods. Bruno > Quoting "Paulo J. Matos" : > > >>Hi all, >> >>I have generated some sudoku based instances. Would those be >>interesting for SMTCOMP? >>Want me to send them in? >> >>Cheers, >>-- >>Paulo Jorge Matos - pocm at sat inesc-id pt >>Web: http://sat.inesc-id.pt/~pocm >>Computer and Software Engineering >>INESC-ID - SAT Group >> > > > -- Bruno Dutertre | bruno@csl.sri.com CSL, SRI International | fax: 650 859-2844 333 Ravenswood Avenue, Menlo Park CA 94025 | tel: 650 859-2717 From tinelli at cs.uiowa.edu Thu Apr 27 16:31:48 2006 From: tinelli at cs.uiowa.edu (Cesare Tinelli) Date: Thu Apr 27 16:32:00 2006 Subject: [SMTCOMP] Sudoku based instances In-Reply-To: <11b141710604270851l16d7270djace5e1a2ef4917db@mail.gmail.com> References: <11b141710604270851l16d7270djace5e1a2ef4917db@mail.gmail.com> Message-ID: <44515464.1020300@cs.uiowa.edu> Hi Paulo, Paulo J. Matos wrote: > Hi all, > > I have generated some sudoku based instances. Would those be > interesting for SMTCOMP? > Want me to send them in? > Thanks a lot for the offer. But I think that the goals of SMT-LIB and SMT-COMP would be better served by different classes of benchmarks, as nicely explained by Bruno in his reply. Best, Cesare > Cheers, > -- > Paulo Jorge Matos - pocm at sat inesc-id pt > Web: http://sat.inesc-id.pt/~pocm > Computer and Software Engineering > INESC-ID - SAT Group > From pocmatos at gmail.com Fri Apr 28 03:37:20 2006 From: pocmatos at gmail.com (Paulo J. Matos) Date: Fri Apr 28 03:37:24 2006 Subject: [SMTCOMP] Sudoku based instances In-Reply-To: <4451061F.2010704@csl.sri.com> References: <11b141710604270851l16d7270djace5e1a2ef4917db@mail.gmail.com> <1146157222.4450f8a6afb59@webmail.colorado.edu> <4451061F.2010704@csl.sri.com> Message-ID: <11b141710604280337h482b64a9tde74559593165269@mail.gmail.com> On 27/04/06, Bruno Dutertre wrote: > Hyondeuk.Kim@colorado.edu wrote: > > Yeah, I think they are interesting. > > > > Hyondeuk > > > > Well, I'm not so sure these kinds of puzzles are particularly > relevant to SMT: In fact I don't think they are relevant for the SMT Competition... I've sent the email initially to know if would be interesting for anyone in this mailing list. > - they can be encoded as a big conjunction (no Boolean structure). Indeed, I agree with you although we can also make the encoding as complex as we wish adding additional redundant information. Still, the benchmarks were generated to test my solver and not to make interesting benchmarks. > - they do not require combination of decision procedures That's not really a problem, right? IDL doesn't require a combination of decision procedures. Only a Boolean enumerator and a decision procedure for difference constraints are needed. > - they are probably easy to solve using CSP or pure Boolean > SAT techniques and encoding them in arithmetic is likely > to be much less efficient. I doubt anybody who really cares > about solving Soduku (if such a person exists) would think > of using SMT solvers for it. I don't think people use solvers, be it CSPs or SAT solvers, in the daily life to solve sudoku anyways. As I said they were interesting at least for me because I can make them as hard as I want and test not only the performance of the solver but also its memory usage and its data structures. > > Because of this, I don't think performance of an SMT solver > on Sudoku is likely to tell us a lot about performance of the > same solver on more realistic applications (e.g., verification > problems). Agree, but not all examples in SMTCOMP are 'realistic' anyways. > We should try to evaluate SMT solvers on problems that > require decision procedures, not on problems that can be solved > faster/better using other methods. > Indeed! Probably I was misunderstood, I don't think they should be used for SMT Solver comparison or evaluation or even to be used for sudoku solving. Just though someone might be interested in them just for hobby, testing or any other non-pratical motif. Anyway, I agree with you when you say that they are not useful for the competition. :) Cheers, Paulo Matos > Bruno > > > > Quoting "Paulo J. Matos" : > > > > > >>Hi all, > >> > >>I have generated some sudoku based instances. Would those be > >>interesting for SMTCOMP? > >>Want me to send them in? > >> > >>Cheers, > >>-- > >>Paulo Jorge Matos - pocm at sat inesc-id pt > >>Web: http://sat.inesc-id.pt/~pocm > >>Computer and Software Engineering > >>INESC-ID - SAT Group > >> > > > > > > > -- > Bruno Dutertre | bruno@csl.sri.com > CSL, SRI International | fax: 650 859-2844 > 333 Ravenswood Avenue, Menlo Park CA 94025 | tel: 650 859-2717 > > -- Paulo Jorge Matos - pocm at sat inesc-id pt Web: http://sat.inesc-id.pt/~pocm Computer and Software Engineering INESC-ID - SAT Group From Hyondeuk.Kim at colorado.edu Sat Apr 29 21:36:43 2006 From: Hyondeuk.Kim at colorado.edu (Hyondeuk.Kim@colorado.edu) Date: Sat Apr 29 21:36:50 2006 Subject: [SMTCOMP] Sudoku based instances In-Reply-To: <11b141710604280337h482b64a9tde74559593165269@mail.gmail.com> References: <11b141710604270851l16d7270djace5e1a2ef4917db@mail.gmail.com> <1146157222.4450f8a6afb59@webmail.colorado.edu> <4451061F.2010704@csl.sri.com> <11b141710604280337h482b64a9tde74559593165269@mail.gmail.com> Message-ID: <1146371803.44543edb78a83@webmail.colorado.edu> I believe there are a lot of disequality constraints in Sudoku problem. It is interesting to check how SMT solver deals with instances that are rich in disequalities. It is true that Sudoku problem can be encoded as a big conjunction. In fact, all difference logic problem can be encoded as a Boolean formula using finite instantiation. I think Sudoku problem is a good starting example to check how the solver deals with a lot of disequalities. There can be more interesting benchmark with more complex boolean structure rich in disequalities, for example, scheduling problem. Thanks, Hyondeuk Quoting "Paulo J. Matos" : > On 27/04/06, Bruno Dutertre wrote: > > Hyondeuk.Kim@colorado.edu wrote: > > > Yeah, I think they are interesting. > > > > > > Hyondeuk > > > > > > > Well, I'm not so sure these kinds of puzzles are particularly > > relevant to SMT: > > In fact I don't think they are relevant for the SMT Competition... > I've sent the email initially to know if would be interesting for > anyone in this mailing list. > > > - they can be encoded as a big conjunction (no Boolean structure). > > Indeed, I agree with you although we can also make the encoding as > complex as we wish adding additional redundant information. Still, the > benchmarks were generated to test my solver and not to make > interesting benchmarks. > > > - they do not require combination of decision procedures > > That's not really a problem, right? IDL doesn't require a combination > of decision procedures. Only a Boolean enumerator and a decision > procedure for difference constraints are needed. > > > - they are probably easy to solve using CSP or pure Boolean > > SAT techniques and encoding them in arithmetic is likely > > to be much less efficient. I doubt anybody who really cares > > about solving Soduku (if such a person exists) would think > > of using SMT solvers for it. > > I don't think people use solvers, be it CSPs or SAT solvers, in the > daily life to solve sudoku anyways. As I said they were interesting at > least for me because I can make them as hard as I want and test not > only the performance of the solver but also its memory usage and its > data structures. > > > > > Because of this, I don't think performance of an SMT solver > > on Sudoku is likely to tell us a lot about performance of the > > same solver on more realistic applications (e.g., verification > > problems). > > Agree, but not all examples in SMTCOMP are 'realistic' anyways. > > > We should try to evaluate SMT solvers on problems that > > require decision procedures, not on problems that can be solved > > faster/better using other methods. > > > > Indeed! Probably I was misunderstood, I don't think they should be > used for SMT Solver comparison or evaluation or even to be used for > sudoku solving. Just though someone might be interested in them just > for hobby, testing or any other non-pratical motif. > > Anyway, I agree with you when you say that they are not useful for the > competition. :) > > Cheers, > > Paulo Matos > > > Bruno > > > > > > > Quoting "Paulo J. Matos" : > > > > > > > > >>Hi all, > > >> > > >>I have generated some sudoku based instances. Would those be > > >>interesting for SMTCOMP? > > >>Want me to send them in? > > >> > > >>Cheers, > > >>-- > > >>Paulo Jorge Matos - pocm at sat inesc-id pt > > >>Web: http://sat.inesc-id.pt/~pocm > > >>Computer and Software Engineering > > >>INESC-ID - SAT Group > > >> > > > > > > > > > > > -- > > Bruno Dutertre | bruno@csl.sri.com > > CSL, SRI International | fax: 650 859-2844 > > 333 Ravenswood Avenue, Menlo Park CA 94025 | tel: 650 859-2717 > > > > > > > -- > Paulo Jorge Matos - pocm at sat inesc-id pt > Web: http://sat.inesc-id.pt/~pocm > Computer and Software Engineering > INESC-ID - SAT Group >