The 5th European Symposium on Validation and Verification of Knowledge Based Systems
June 9-11, 1999Oslo, Norway
http://www.dnv.no/research/safekbs/eurovav99/eurovav99.htm
|
The 5th European Symposium on Validation and Verification of Knowledge Based SystemsJune 9-11, 1999Oslo, Norway http://www.dnv.no/research/safekbs/eurovav99/eurovav99.htm |
||
|
|
||
EUROVAV'99 took place in Oslo this year. According to my guide book Oslo is "a vibrant, self-confident city whose urbane, easy going air bears comparison with any other European city" (1) --- good!. The guide book goes on to describe a stereo-typical Norwegian as "a healthy, hearty figure in a wholesome woolly jumper"! The city has a number of world famous museums, of note are the Edward Munch museum and the Viking Ship Museum, the Vigeland Park is also worth a visit.
Man kicking a dog in Vigeland Park! |
Food and drink wise Oslo is expensive, particularly so in the case of the latter (half a litre of beer costs somewhere between 40NOK to 50NOK). The guide book explains this phenomena as follows "For reasons that remain obscure - or a least culturally complex - many Norwegians can't have a drink or two but have to get absolutely wasted. The majority of their compatriots deplore such behaviour and have consequently imposed what amounts to alcoholic rationing: thus, although booze is readily available ... it's taxed up to the eyeballs". As a consequence many EUROVAV delegates were forced to significantly change the usual habits! Those of us who did venture out found that the most popular bars were in and around the Aker Brygge waterfront area and at the back of the Radhus (my guide book warns that these bars are "assertively heterosexual!"!). |
The conference itself took place in the Soria Moria hotel where most of the delegates also stayed. The hotel is situated some 10km to the north of Oslo in a upland forest region known as the Nordemarka. The hotel offered superb views of the surrounding forests and hills stretching away to the horizon. The food was excellent (particularly the sea food) and delegates were able to make use of the pleasant surroundings by taking walks in the forest (when it was not raining) and enjoying the indoor swimming pool and sauna (regardless of whether it was raining or not). The conference facilities were ideal (they included features such a screens that descended from the ceiling and curtains that closed at the touch of a button). A special mention should be made here acknowledging the help of Det Norske Veritas (DNV) without whose support this excellent venue would not have been possible, and to Anca Vermesan the conference co-chair and local organiser.
Another nice touch was that each delegate, on entering their room, found that there was a message on Norwegian television welcoming them to the country! A more problematic feature was that the room doors opened outward which caused difficulties for some of the more serious academics who had trouble getting out of their hotel rooms as a result!
As in previous years, EUROVAV'99 aimed to bring together researchers and developers from a wide range of academic, research and industrial backgrounds. There were 30 participants from Belgium, France, Netherlands, Poland, Portugal, Spain, Italy, UK, USA and Norway. The delegates have had the opportunity to discuss their ideas with research staff working at the leading edge of knowledge based system developments and with developers and certifiers with experience of the practical issues of getting and maintaining systems in operation.
The symposium successfully integrated two contrasting streams. On one side, the theoretical stream presented the best of recent developments in V&V of KBS, covering a wide range of technical areas. On the other side, the applications stream presented papers describing practical applications of KBS and the associated V&V approaches. For each stream there were keynote addresses by acknowledged international experts in their field. For the theoretical stream, Trevor Bench-Capon from University of Liverpool, addressed the role of ontologies in the Verification and Validation of KBS. His presentation provided an excellent start point for the symposium. Following the application stream, one of the highlights of this event was a seminar centred around software product certification. There was a guest speaker from the US, Jeffrey Voas, talking about user participation-based software certification, and a DNV invited speaker, Per Martinsen, talking about certification activities of software/knowledge based components integrated within programmable systems. The presentations were all followed by lively and fruitful discussions.
The scope of EUROVAV has expanded over recent years to cover not just traditional V&V of KBS, but to include other software and artificial intelligence technologies, as well as application areas. This year's symposium included papers organized under the following headings:
Application papers covered domains as diverse as resource management, medicine, transmission networks and maritime. All papers were rigorously refereed to ensure that those selected met the needs demanded by delegates and the quality required for publication. The symposium proceedings are published in a book form by Kluwer Academic Publishing (Utrecht, Netherlands).
3. DAY ONEThe symposium was opened by Professor Pal Bergan, director of research for Det Norske Veritas (DNV), who welcomed us all and made a fitting joke about the rain. Pal Bergan then went on to talk about the significance of V&V generally and in the context of DNV. The main point being that "computers are part of systems therefore we should be concerned with the effect of computer failure to the larger system". Pal Bergan then went on to describe the strategic research vision of DNV (with a 20 year perspective) commenting that the linear R&D process was moving to a cyclic model" --- the aim being to shorten R&D time scales. The talk provided an excellent start point for the interchange of ideas and discussion that was to take place over the following three days. |
Pal Bergan, director of research for Det Norske Veritas (DNV). |
Trevor Bench-Capon building "roads"!. |
The conference included a number of invited talks the first of which was by Trevor Bench-Capon from the University of Liverpool whose talk was entitled "The Role of Ontologies in the Verification and Validation of KBS". The first half of the presentation was concerned with the motivation of ontologies --- "an explicit specification of the conceptualisation of a domain". Trevor Bench-Capon emphasised that an ontology was not a model, "an ontology defines classes of concepts while models include instances of concepts". He then went on to state that ontologies describing KBS can be validated; and introduced a software tool, PRONTO, to support this approach to KBS evaluation. There were a number of questions. Barry Eaglestone wondered whether the approach described would work with "ready made" ontologies such as those used in health domains. Trevor Bench-Capon responded that the nature of the ontology was not significant as long as all relevant parties agree to subscribe to it (i.e. to agree to a mutual understanding). Trevor Bench-Capon is a well known international figure in the field of KBS V&V and in the maintenance of such systems. As the symposium continued the ontologies concept was to become a central theme which delegates returned to repeatedly. |
After a short break to allow the smokers to rush outside to appease their cravings, Trevor Bench-Capon's invited talk was followed by two sessions on Knowledge Revision and Refinement, one before lunch and one after. The first session was chaired by Frans Coenen (the other conference co-chair) and featured two speakers.
|
The first presentation was given by Jan Vanthienen of the Katholieke University of Leuven whose talk was entitled "Illustrating knowledge Base Restructuring and Verification in a real world application". Jan Vanthienen is an internationally recognised expert on decision tables especially in the context of KBS. He described a decision table modularization approach to restructuring KBS. Modularization is an important concept with respect to verification, validation and maintenance, as it encapsulates knowledge into isolated modules which are easier to verify and maintain. By avoiding redundant specification of knowledge, the risk of introducing inconsistencies during maintenance is minimized. He also pointed out the difference between various decision table approaches ("multiple-hit" versus "single-hit" tables), a distinction which is often overlooked both in research and practice, leading to incorrect assessment of the technique. |
Jan Vanthienen making decisions about tables and tables about decisions. |
Jan Vanthienen illustrated his talk with a specific KBS application (SEAR --- Enterprise Human Resources Advisory System), the aim of which is to optimise human resource management in a social and juridical consultantship setting based on the Spanish Social Security General Regime Law. He then went on to describe how the SEAR KBS could be restructured into a decision table hierarchy and consequently verified and validated using a decision table tool, PROLOGA, developed by researchers at K.U.Leuven.
At the end of the talk Frans Coenen asked a question concerning the storage structure used in PROLOGA, Jan Vanthienen responded that they could be thought of a tables in a database. Continuing in this theme Barry Eaglestone commented that the restructuring approach was not unlike the normalisation technique used by some database developers, Jan Vanthienen acknowledged that this was true but warned that parallels between decision tables and database tables should not be taken too seriously. Finally, Mieczyslaw Owoc suggested that to achieve consistency in a KBS alternatives should be presented to the developer and that perhaps this should be taken on board by the PROLOGA team.
The second presentation was by Robin Boswell of the Robert Gordon University whose talk was entitled "Organising Knowledge Refinement Operators". This talk was the first of two talks concerning the KRUSTWorks project (the second was presented in the afternoon by Nirmalie Wiratunga). KRUSTWorks is a follow on from the KRUST project which in turn has its roots in a number of European V&V projects undertaken in the late 1980s and early 1990s. Robin Boswell described a number of generic KRUSTWorks refinement tools under development (existing tools are only applicable to particular environments and/or implementation languages). Central to this "tool kit" is a set of generic refinement operators, and a representation language comprising a knowledge hierarchy to describing the rule-elements within a KBS. Interestingly the latter may be regarded as an ontology for rule-elements, Robin Boswell thus provided a link back to Trevor Bench-Capon's presentation, however in this case the ontology is used to "tell" a KRUST tool which refinement operators to apply to which element. The approach had been tested using a number of forms of representation; the talk concentrated on an evaluation using using Logica's PFES (Product Formulation Expert System) shell.
Robin Boswell busy reorganising his knowledge refinement operators. |
On completion of the presentation Antoni Ligeza questioned whether the approach could really address the many rule base formats available such as fuzzy and temporal rule sets, and also take into consideration rules that are hidden within the inference engine such as conflict resolution rules. Robin Boswell responded that the KRUSTWorks representation of the reasoning process could handle conflict resolution, and that other aspects such as temporal rules might be addressed (although he doubted that the research team would get round to considering fuzzy rule structures). The third talk in this section (in the after lunch session chaired by Trevor Bench-Capon) was by Nirmalie Wiratunga, also of the Robert Gordon University, and described the overall operation of KRUSTWorks continuing from Robin Boswell's presentation of the morning. Nirmalie Wiratunga's talk was entitled "Incorporating backtracking search with knowledge refinement", and explored three strategies for revising a Rule Base: |
Essentially the approaches presented were iterative in nature and operated using constraint satisfaction techniques. Frans Coenen wondered what would happen given a training set that could not be resolved by the CSP technique, i.e. (Trevor Bench-Capon) would the system create a new rule. Robin Boswell replied on behalf of Nirmalie Wiratunga that the system only refined existing rules and did not create new ones although they had looked at the possibility of "splitting" rules.
The final talk in this section was presented by Stephan Loiseau of the Universite d'Artois in France, and was entitled "Validation and refinement versus revision" and revolved around the distinction/overlap between KBS validation and KBS revision/refinement (maintenance). This talk described some preliminary work to bring together techniques for revision and validation/refinement. Stephan Loiseau defined refinement as the "modification of an incoherent KB in order to restore its coherency", and revision as the modification of an inconsistent KB to restore consistency. A coherent rule base is then one which "if and only if with any valid input, contradictory results cannot be inferred". He then went on to state that revision requires a model defined by an expert, and that some mechanism involving costs and penalties was required so that choices can be made between alternatives. Stephan Loiseau suggested a number of possible approaches (three) to address this. At the end of the talk there was much discussion (from Trevor Bench-Capon, Rik Gerrits and Antoni Ligeza amongst others) concerning the selection criteria suggested by Stephan Loiseau. There was also discussion concerning the distinction between revision and refinement --- "you revise a single fact, but refine a set of facts (to be consistent)", i.e. a one to many distinction.
3.2. CertificationThe final session of the day was concerned with software certification and was chaired by Anca Vermesan. The first presentation in this section was an invited talk by Jeffrey Voas co-founder of Reliable Software Technologies (USA). The title of Jeffrey Voas' presentation was "User Participation-based software certification". In his talk Jeffrey decried the use of popular approaches to software certification founded on publisher oaths and certification auditors (who carry out spot-checks to confirm such oaths). Jeffrey Voas maintained that this approach was seriously flawed --- "Dirty water can run from clean pipes", and then went on to propose a certification methodology that does not employ auditors and publisher oaths but was founded on end user groups, i.e. independent product certification. |
Jeffrey Voas, demonstrating how dirty water can run out of clean pipes. |
At the end of the talk Frans Coenen commented that the proposed use of end user groups would require a significant infer structure to support it, Jeffrey Voas agreed. Jan Vanthienen and Trevor Bench-Capon both noted that the testing that Jeffrey Voas was suggesting checked only for robustness and not correctness of content. Jeffrey Voas agreed again, but commented that robustness was only a start point, "robustness is the best we can do, after that we need to establish what technologies are out there that can be used for certification". Jeffrey Voas is a well known practitioner amongst the more traditional software testing community and has written two books on the subject.
The second talk, also invited, was presented by Per Martinsen, principal surveyor and head of the "Control Systems and Nautical Safety" department at DNV, who talked about "Software certification activities for programmable systems". The talk described the certification process in the wider context especially with respect to the practice of DNV. Jan Vanthienen asked a question concerning what was being certificated, Per Martinsen replied that what they are doing is "type approval of application and not correctness". Jeffrey Voas noted that Per Martinsen claimed a 75% reduction in premiums as a result of certification, "they must trust the certification authorities in a way that do software industry does not". Frans Coenen responded that DNV's certification process has been built up over 200 years experience while the certification that the software engineering industry wished to do has zero years experience. Jeffrey Voas agreed with "the importance of historical data". Anca Vermesan also considered that historical data was important, but added that in the shipping industry the desire for certification originated with the ship owners, she was not convinced that the same desire existed amongst the owners of software systems, KBS or otherwise.
The final talk of the day was by the conference co-chair Anca Vermesan (of DNV) the title of which was "Verification and Validation in Support for Software Certification Methods". The talk highlighted the role of KBS technology in the certification process and also described links with the Safe-KBS project completed in 1998 (see below)). The main point of the talk was that "certification is the most advanced use of KBS V&V techniques". Anca Vermesan is a well known and respected personality within the KBS V&V community especially for her recent work on certification.
Anca Vermesan (EUROVAV'99 co-chair) supporting software certification methods. |
At the end of the talk Jeffrey Voas queried whether KBS could really be used for safety critical systems "are they not too slow?". Anca Vermesan agreed that that this was a problem, but it was not a problem applicable to just safety critical systems but also to any system where response times are important. Trevor Bench-Capon noted that the nature of KBS such was that often the average case performed acceptably, it was the "worst" cases that caused problem and that this variance must cause a further problem for certification. Antoni Ligeza pointed out that there are many real time expert systems around and cited KHEOPS (developed at LAASD du CNRS, Toulouse, France) which was used on the TIGER project. Rune Steinberg noted that they did look at KBS and safety critical aspect in the Safe-KBS project and that he would return to this subject during his presentation. Mieczyslaw Owoc commented that he was still unsure about the definition of validation in the context of certification. |
Day one ended with a drinks reception overlooking the stunning views from the hotel bar's veranda.
Day two comprised five sessions, two of which (before lunch) were devoted to tools and techniques. The presentations after lunch covered applications, knowledge management and "the wider perspective".
The first of the two tools and techniques sessions was chaired by Frans Coenen. The session commenced with a general context setting presentation given by Mieczyslaw Owoc of the Wroclaw University of Economics. Mieczyslaw Owoc presented a slightly unorthodox view of KBS V&V with one considered to be a sub-component of the other. Mieczyslaw Owoc's principal concern was that of validating knowledge (rather than the structure of KBS), and in this context he suggested a number of guiding principals.
The second presentation in this section was given by Eric Gregoire of the Universite d'Artois, and was entitled "Progressive instantiation for the logical validation of nonmonotonic KBs". The talk concentrated on the possibility of a first order non-monotonic KBs containing contradictory knowledge. Eric Gregoire noted that the checking of logical consistency of rule bases expressed in a clausal propositional logic form is an NP complete problem. To tackle this problem Eric Gregoire suggested that consistency checking should be performed incrementally during the development of a KB rather than on completion, and then went on to propose a mechanism to achieve this based on a depth-limited instantiation schema. Frans Coenen asked whether Eric Gregoire had any measures with which to assess the claimed computational efficiency of his approach. Eric Gregoire replied by noting that they had tested KBs with as many as 100,000 clauses, and that this had not "overtaxed" the approach. Trevor Bench-Capon noted that Eric Gregoire made the assumption that clauses are order and asked whether this was significant or note. Eric Gregoire stated that it was not significant with respect to the consistency checking issue although it can simplify the inference process. Antoni Ligeza commented on problems finding conflicting sets suggesting this was akin to model based reasoning.
|
After a short break for coffee the session recommenced (under the auspices of Antoni Ligeza who chaired the second half of the session) with a joint presentation (double act!) by Silvie Spreewenberg and Rik Gerrits from LibRT Redeployment Technology (Netherlands). Silvie Spreewenberg and Rik Gerrits presented a tool to verify KB using meta-rules to be used within the AION object-oriented KBS development environment (where facts are attributes and rules are methods, both of which can be inherited). The tool is designed to be used during and after the development of KBSs and focuses on logical verification. The tool uses meta rules, meta information and the inference engine of Aion to accomplish this task supported by an end user editor. By using the same inference engine in the verification process as the inference engine used to evaluate and fire the rules to be verified Silvie Spreewenberg and Rik Gerrits maintained that there can be no discrepancy between the run time logic and the logic used in the validation process. |
Rik Gerrits in search of Lars Pedersen. |
At the end of the presentation Anca Vermesan asked, "on a practical note", whether object orientation solved some of the problems of concern to the KBS V&V community. Rik Gerrits felt that it did not, in fact it made things harder. Frans Coenen thus wondered what advantage object-orientation had in the context of KBS. Rik Gerrits responded that the technology produced better KBS more efficiently (but harder to test). Rune Steinberg asked for further details concerning end user editors for KBS, what was their experience? Silvie Spreewenberg responded that the current system was only an alpha version but that they hoped to start collecting end user feed back soon. The role of object-orientation in the development of KBS and their V&V was to become another theme of EUROVAV'99.
As a side issue it is worth noting that Silvie Spreewenberg and Rik Gerrits were also on a quest to find one Lars Pedersen, a musician and suspected artist, some of whose paintings they were interested in buying. Eventually, having tracked down not only the great man himself, but also his best friend and girl friend they discovered that he was in fact not an artist and thus had no paintings to sell!
Juliette Dibie "thinking in pictures". |
The final talk in the tools and techniques session was presented by Juliette Dibie of University of Parix IX Dauphine. Juliette Dibie's talk was entitled "Constraints for validation of conceptual graphs". The foregoing presentations had all concentrated on various established rule representations (with the possible exception of the object oriented approach suggested by Silvie Spreewenberg and Rik Gerrits), in contrast Juliette Dibie consider the semantic validation of conceptual graphs based on constraints given by experts for validation purposes. Juliette Dibie commenced with an overview of the conceptual graph model she had adopted and then went on to consider both the existential and descriptive constraints applicable to this model. Juliette Dibie completed the talk by presenting a specification for a semantic validation of KB founded on conceptual graphs (implemented on the CoGITo platform). Using this approach a KB is then semantically valid if it satisfies all the given constraints (Juliette Dibie used the term "specification" to describe such expert knowledge given for validation purposes). |
On completion of the presentation Michael Schroeder queried the cost of using a graphical representation. Juliette Dibie admitted that there was a cost overhead involved, but it did make the KB more understandable. Trevor Bench-Capon disagreed with this (Trevor Bench-Capon maintains that people do not "think in pictures" --- a view not subscribed to by everyone). Rik Gerrits asked for more detail concerning CoGITo --- it is a tool box for manipulating conceptual graphs marketed by the University of Montpelier. Antoni Ligeza commented that conceptual graphs are good for knowledge representation but problems arise when superimposing a reasoning mechanism over the graph --- the graph must be transposed into some form of logic.
The first session after lunch was concerned with presentations examining the application of KBS V&V and was chaired by Barry Eaglestone.
The first presentation, by Francisco Ribeiro from the Instituto Superior Tecnico in Lisbon, concerned the V&V of a KBS integrated with a case based reasoning system for supporting human experts in assessing applications for the house renovation grant system in Portugal. Francisco Ribeiro commenced by considering the development of the system (called HRGS) and then went on to consider the approach taken to verify and validate this system.
|
At the end of the talk there was some discussion (Frans Coenen) concerning the application and how the linkage between the KBS and the case base operated Nirmalie Wiratunga raised concern about the possibility of the case base containing duplicates. Trevor Bench-Capon wondered why the case base contained only positive cases and no negatives (cases where approval had not been given). Mieczyslaw Owoc also requested further clarification. Francisco Ribeiro responded that the nonexistence of a case in the case base indicated negation. If this was so Nirmalie Wiratunga wondered how Francisco Ribeiro handled situation where there was not a precise case to match the current case. Francisco Ribeiro replied that adaptation problems were addressed by facilities used in the KapaPC tool kit used to implement the system. This was followed by further discussion concerning the matching of cases with Antoni Ligeza noting that numeric matching was not enough, it was also necessary to support symbolic matching. The session chair, Barry Eaglestone, was eventually forced to curtail the discussion. |
Francisco Ribeiro fusing KBS and CBR. |
The second presentation (described by Barry Eaglestone as being "highly charged", "electric" and guaranteed to "send sparks flying") was entitled "Validation and verification of knowledge-based systems for power system control centres". The talk was presented by Jorge Santos of the Polytechnic Institute of Porto despite the fact that his arm had been temporally bolted together while he recovered from a football injury (Jorge Santos is also a keen advocate of Liverpool Football Club). Jorge Santos' presentation focused on the V&V of a KBS (called SPARSE) used to assist operators of Portuguese Transmission Control Centres in incident analysis and power restoration. Although this was the central theme of the presentation Jorge Santos also provided a more general insight into the application of KBS V&V techniques. Jan Vanthienen asked a question concerning the redundancy checking mechanism used, and Antoni Ligeza asked a question concerning the representation of temporal constraints within the system ---- "points or intervals?".
The third talk in this session was given by Florence Sellini of PSA Peugeot Citroen (France), the title of which was "A priori verification of product models in mechanical design". Florence Sellini presented a discussion of the verification of product modelling for design aid systems for mechanicals sets based on the knowledge of experts. To this end Florence Sellini proposed a particular knowledge representation which included additional constraints intended to conform to the represented knowledge. The talk provided an extremely useful demonstration of "what is/can be/must be done in a realistic KBS application". Anca Vermesan asked some questions concerning the representation of knowledge within the system and the standardisation of the representation. Further, as a consequence of Florence Sellini's talk, Anca Vermesan thought that she might by a Peugeot next time round!
Michael Schroeder verifying business processes. |
The final talk in this session was given by Michael Schroeder of City University London. Michael Schroeder hails from Germany and has recently moved to City University from British Telecom (BT); the work described in his talk, "Verification of Business Processes for a Correspondence Handling Venter using CCS" (Calculus of Communicating Systems), was carried out at BT. Michael Schroeder's presentation was concerned with process automation commencing with the observation that although work has been done on the animation and simulation of such processes (validation) no work has been done on their verification, i.e. to identify anomalies such as deadlock, livelock and safety & liveness. The main contribution of Michael Schroeder's presentation was the link between verification tools and process definitions expressed in the Process Interchange Format (PIF). The approach allowed users to check for liveness and safety properties such as ensuring that jobs are possibly fixed and necessarily dispatched. Furthermore Michael Schroeder showed how checks for deadlocks and livelocks could be made using the algebra. |
At the end of the presentation Trevor Bench-Capon asked whether "it would not make more sense to start with CCS rather than PIF", Michael Schroeder responded that the reason for opting for CCS was that it was more generic in that it could also be used with other interchange formats. Frans Coenen wondered how the sequencing of events was checked. Michael Schroeder commented that this was more the concern of modelling techniques than the tools described. Rune Steinberg wondered if Michael Schroeder had compared his results with those of other researchers. Michael Schroeder admitted that he had not, but acknowledged that this was clearly desirable.
No modern expert systems or KBS conference would be complete without a session on Knowledge Management (KM), the EUROVAV team where therefore determined not to be signalled out as an anomaly for not including a KM session. The session was chaired by Trevor Bench-Capon and featured two contributions.
The first presentation, by Henrik Giaever, of DNV was entitled "Verification and Validation in Support of Knowledge Management". This was a very theoretical presentation. Henrik Giaever commenced by stating that KM is "to do with concepts, methods and tools for managing intellectual capital". This was then followed by a philosophical discussion on the nature of knowledge concentrating on different views of information (consciousness). Henrik Giaever illustrated his talk with a number of slides showing images which could be interpreted in different ways (cube, old woman/young girl --- "what do you see?").
The second talk was given by Robert Engels of CognIT (Norway) --- he is actually a Dutchman, but lives in Norway. The title of Robert Engels' presentation was "CORPORUM --- An Automated Ontology Construction Tool". Robert Engels' talk focussed on the need to support knowledge companies who are in the business of capturing knowledge for KM. The talk included a demonstration of the CORPORUM tool which used the notion of an ontology to extract knowledge. Robin Boswell commented that he would have liked to have seen the ontology itself, unfortunately this was unavailable. Both Trevor Bench-Capon and Anca Vermesan also asked about the nature of the links.
4.4. The Wider PerspectiveThe final session of day two of EUROVAV (also chaired by Trevor Bench-Capon) was concerned with the "wider perspective". The aim of this session (and a similar session on the following day) was to explore the wider application of KBS V&V techniques. The first presentation was by Frans Coenen of the University of Liverpool who explored the application of KBS V&V techniques to the database domain. Frans Coenen commenced by identifying overlap between KB and database systems and then went on to suggest a number of future directions for research to investigate possible "cross-over" technologies. Of note were the the opportunities presented by the ontologies concept and the use of the object-oriented paradigm. Jan Vanthienen felt that a further area for investigation might be the automated gathering of data using KBS techniques, Barry Eaglestone agreed with this. Anca Vermesan welcomed Frans Coenen's attempts to widen the scope of KBS V&V. (Frans Coenen was one of the two co-chairs of EUROVAV.) |
Frans Coenen examining his wider perspectives. |
Mieczyslaw Owoc and Antoni Ligeza, poles apart! |
The second presentation (and last of the day) was by Antoni Ligeza of the Institute of Automatics in Cracow who gave a presentation on "Intelligent data and knowledge analysis and verification: towards a taxonomy of some specific problems". This continued on the theme of Frans Coenen's talk in that it focussed on the analysis and verification of data and knowledge (D&K) using a common model founded on an extension of the relational database model. Antoni Ligeza concentrated on the meta-level analysis of D&K tables to identify problems associated with data representation, sufficiency, internal and external constraints, correctness, equivalence etc, but also considered various technical problems encountered when implementing the approach. The motivation for the talk was that "people trust DB technology, they do not trust expert systems", "you can prove anything in logic, but algebra is still more trustworthy". Antoni Ligeza also stated that he would like to "loosen the tight corset of verification" (!) and suggested some additional problems which we might wish to consider, including problems associated with the interoperability of multiple KBS and other data sources. |
At the end of the talk there was some discussion (Anca Vermesan and Frans Coenen) concerning the nature of the specification algebra used by Antoni Ligeza and further extensions to his "multiple" approach. Antoni Ligeza is a regular attender at EUROVAV and related KBS V&V workshops.
Day two ended with the conference banquet held within the hotel confines, followed by drinks on the veranda while watching the sun set over the superb scenery (it had stopped raining --- briefly!).
The final day of EUROVAV continued with the theme of the wider perspective from the end of the previous day with a similar session chaired by Robin Boswell. The second session of the day was concerned with safety critical systems in general and the Safe-KBS project in particular. The conference ended with a final tools and techniques session.
5.1. The Wider Perspective ContinuedThe first presentation of the day was by Barry Eaglestone of the University of Sheffield and continued on the database theme (introduced by Frans Coenen and Antoni Ligeza on the previous day) but from the DB perspective. Barry Eaglestone described some of the issue of concern in the database community in the context of schema integration of federated databases, and also briefly overviewed a semi-automated methodology for the schema analysis phase of schema integration based upon the ontology-based conceptualisation of schema semantics. (Ontologies are traditionally referred to as "common concept models" in the database community a phrase which at least conveys some meaning!) |
Barry Eaglestone still exploring wider perspectives, but now wearing a wholesome woolly jumper (incidently, just so that nobody gets the wrong idea, that is a bottle of lemonade he is holding). |
Barry Eaglestone ended his talk, by popular demand, with his famous "cowboy" slide (see below). There was some discussion afterwards (Frans Coenen) concerning the nature of federated databases - "How many databases are we talking about?". Barry Eaglestone was co-chair (together with Frans Coenen) of the VVI workshop held as part of DEXA'98.
The second talk in this session was by Frode Hogberg of DNV (who had incidentally commandeered the control panel for the curtains, lights, screens etc. on day one of the conference; thus preventing other delegates from playing with it). Frode Hogberg considered the "Applicability of Conventional Software V&V to Knowledge Based Components". Frode Hogberg presented an experimental framework, to determine whether a technique is applicable or not, based on concepts taken from ideas about mutation testing. The framework itself consists of a number of steps guiding the researcher/practitioner in the assessment process. Mutation testing is used to simulate faults in an example program to determine the technique's ability to detect them. Frode Hogberg suggested that the framework provided a basis for qualitative assessment of the applicability and efficiency of applying specific traditional V&V and testing techniques to KBS components. To illustrate these ideas Frode Hogberg seeded a sample KBS with faults through a mutation process, and then use a number of traditional V&V methods to find these faults. Rik Gerrits wondered whether Frode Hogberg's framework could be used to determine the growth of a KBS, would it become too slow?. Anca Vermesan, responded on behalf of Frode Hogberg and emphasised that it only produced a qualitative result. Robin Boswell wondered if the frame work checked every line in a KBS (i.e. coverage). Trevor Bench-Capon commented that coverage is not enough, just because every rule is executed does not mean that there is not a fault in the KB. Anca Vermesan added that mutation testing has also been used to "test the usefulness of the test sets".
The safety critical systems session, chaired by Anca Vermesan, comprised two presentations both describing work carried out as part of the "Safe-KBS" ESPRIT project, the aim of which was to define development and certification methodologies for KBS, especially embedded KBS used in safety critical domains. The particular result of this work has been the definition of a "Safe-KBS" life cycle, designed so that it can be integrated with formal methods and provide appropriate support for software certification methods. EUROVAV has over the years traditionally been a good setting for delegates and companies to disseminate results from high profile European projects, such as VIVA, VITAL and VALID. This year's EUROVAV emphasizsed the SafeKBS project, belonging to the Esprit IV Program of the European Community. In this context, the research groups from ENEL/CISE, Italy and Computas AS and DNV from Norway, all SafeKBS partners, reported results from their work within the project.
Giovanna Dondossola enjoying an informal moment away from her formal methods. |
The first presentation in this session was given by Giovanna Dondossola from ENEL/CISE (Milan). The presentation was concerned with "Formal Methods for the engineering and certification of safety-critical Knowledge Based Systems". Giovanna Dondossola commenced by providing an overview of the results of the "Safe-KBS" project and the role of formal methods within the project. This was then followed by a detailed discussion on the requirements for KBS formal methods within the context of both specification and V&V. Giovanna then continued to consider the impact of KBS formal methods on the KBS life cycle pointing out that life cycle models should be defined independently of specific development methods and techniques and went on to consider the elements of such a life cycle based on that developed as part of the Safe-KBS project. |
Finally Giovanna Dondossola gave consideration to the selection and evaluation of KBS methodologies, concluding that no optimum formal method exists appropriate to the full spectrum of KBS development requirements. Trevor Bench-Capon asked why Giovanna had classified Z as a formal method having a "high level of rigour", whilst OBJ3 as a low one. She responded that OBJ3 does not support automated V&V, "there do not seem to be any tools designed to operate with OBJ3". Incidentally if you are ever in Milan Giovanna Dondossola is part owner of a very good vegetarian restaurant!
|
The second SAFE-KBS presentation was given by Rune Steinberg of Computas (Norway). The title of the talk was "Design patterns for safety-critical knowledge based systems". Rune Steinberg argued that, despite the fact that a well designed life cycle will contribute substantially to the confidence in a KBS, there is also a particular need to concentrate on design. Rune Steinberg then went on to show how the Safe-KBS life cycle is supported by a catalogue of design patterns for safety critical KBS which describe design solutions that seek to increase the confidence of using KBS technology. At the end of the presentation Jan Vanthienen wondered whether there was any relation between an architect pattern and a design pattern. Rune Steinberg responded that he had yet to investigate this. 5.3. Tools and Techniques Two |
Rune Steinberg contributing to user confidence in KBS. |
The final session of EUROVAV was a second tools and techniques session (chaired by Anca Vermesan) and comprising a single presentation by Luis Laita from the Universidad Politecnica de Madrid. The title of Luis Laita's talk was "Computer Algebra Based Verification and Knowledge Extraction in RBs: Application to Medical Fitness Criteria". Luis Laita described an algebra that can be applied to many-valued logics, which has been incorporated into a software system called CONSIST written in the computer algebra language CoCoA. Luis Laita commenced by summarising the main ideas of modelling propositional many valued logic with modal operators using polynomial rings over appropriate fields.
Luis Laita juggling Polynomial Rings, Normal Forms and Grobner Bases. |
Luis then demonstrated that the crucial consequence of this construction was the fact that tautological consequence and consistency can then be expressed by the properties of ideals of the corresponding polynomial rings. Given a KBS, the system translates the rules and facts contained in the KBS, together with details of integrity constraints and any other additional information, into polynomials. The "Normal Form" and "Grobner Basis" are then calculated; if these are 0 and 1 respectively the KBS is considered to be consistent, otherwise it is considered to be inconsistent. Luis Laita completed his talk with a demonstration of his software. At the end of the demonstration Trevor Bench-Capon commented that there are typically several ways whereby rules can be removed to restore consistency, and wondered if Luis Laita made use of all these ways. Luis responded that the system demonstrated was only a prototype. Some further discussion followed regarding efficiency (Anca Vermesan) and inconsistent rule sets (Trevor Bench-Capon) which had to be curtailed due to time constraints. |
The conference was closed by Frans Coenen who thanked the contributers, delegates and programme committee for their support, and especially DNV and Anca Vermesan who made it all happen.
Frans Coenen then went on to consider some options for EUROVAV 2001 suggesting that there was a general consensus that it should be held in Germany, and that the venue for EUROVAV 2003 would be in Poland. Frans Coenen also proposed that the scope of the symposium should be widened to encompass V&V issues associated with all AI systems, citing intelligent agents and case base reasoning as appropriate domains. The latter was generally supported by most delegates, but not all, Rik Gerrits felt that the scope should be tightened rather than widened so that only "true" KBS V&V was represented --- the debate goes on!
It was also suggested (Jan Vanthienen) that for future EUROVAVs it might be an idea to include one or two tutorials, especially if it was intended to widen the scope. This idea was also well supported.
Overall EROVAV'99 was a very successful event. The main themes of the conference were:
EUROVAV'99 provided a valuable opportunity to keep up to date with developments in V&V of KBS and to share experiences in the practical issues of developing and certifying knowledge based systems in particular, and software systems in general.
The proceedings are published by Kluwer Academic Publishing (2). Anyone wishing to obtain a copy should contact Anca Vermessan (Anca.Vermesan@dnv.com) ay DNV.
Created and maintained by
Frans Coenen.
Last updated 16 September 2000