Evaluation of the Competence Centre for
Information Systems for Industrial Control and Supervision, ISIS
Linköping University, Linköping
1. Preface, Methodology, and Acknowledgement
On Friday, September 22, two of us, the scientific experts of the evaluation team, Michel Gevers and Lino Guzzella, were briefed by the technical staff of the Centre for Information Systems for Industrial Control and Supervision, ISIS, on their 5 establhed activity areas:
First, the Director, Prof. Lennart Ljung, gave us an overview of the 6 university teams and the 9 companies involved in ISIS, the strategic goals of the Centre and the main means to achieve them, as well as an enumeration of the 11 presently active projects.
This introduction was followed by a 5-minute presentation of each project by a spokesperson of these respective projects. The rest of the day consisted of a sequence of poster presentations, one for each project, some of whom were accompanied by laboratory visits and demonstrations. At each of these poster presentations, we had ample opportunity to discuss with the ISIS team members active in this project, including a number of people from the companies involved. These presentations gave us an excellent global picture of the activities and projects that take place within ISIS. The day ended with a one-hour general discussion at which most of the PhD students involved in ISIS projects were present, as well as a number of industrial partners.
On Monday, September 25, the entire evaluation team, made up of the scientific experts and the two Competence Centre experts, John S. Baras and Per Stenius, had a 4-hour meeting with the Director, the Board, and the Reference Group of ISIS, as well as the President of Linköping University, Bertil Andersson, and the Dean of Engineering, Mille Millnert. We heard presentations about the strategic views of the Centre, some of its scientific and technological results, the industrial benefits, as well as the vision of the university hierarchy about ISIS.
We would like to thank the whole ISIS team for the superb efforts they made in preparing these two days of briefings in such a way that the transfer of information was as efficient as possible. It is also worth pointing out that the atmosphere that prevails within ISIS allowed for very open and frank expressions of opinions during the discussion. In particular we thank the Director, Prof. Lennart Ljung, for masterminding a very efficient and productive use of our time, and the Secretary, Ulla Salaneck, for her superb handling of the administrative details and logistics. We wh to also thank Karl-Einar Sjödin, and Staffan Hjorth of NUTEK, for their thoughtful arrangement and co-ordination that facilitated our evaluations.
2. Development as a Competence Centre. Added Values.
Long-term Strategies and Progress of the Centre
The management of ISIS has a clear view of their strategy as a NUTEK Competence Centre. The key goal, which is obviously shared by both the University staff and the industrial partners, is to maintain a good balance between scientific excellence and industrial relevance. This means that, in particular, the industrial partners fully accept that the projects should be of such a nature and that they should be run in such a way that they lead to PhD theses. This understanding and acceptance by the industrial partners has been cited several times as a key element, in that it allows the discussions to focus on the scientific/technical aspects rather than on organisational matters.
The Board of ISIS has made a clear strategic decision to give preference to establhing a high quality and long-term working relationship with a small number of carefully chosen companies that adhere to the goals mentioned above, rather than trying to expand the number of industrial partners at the cost of diluting these goals. In particular, it has been noted that none of the present industrial partners in ISIS are in competition with each other. This is a clear advantage in terms of trust, cohesion and a free exchange of information. As a funding organisation, NUTEK might question the reluctance of ISIS to increase the number of its industrial partners. We feel that it is probably better to do an excellent job with a small number of companies than to aim for growth at all cost.
As for the scientific content of the Centre’s activities, these have been centred from the beginning on the five areas indicated above. However, the collaboration with the industrial partners has clearly highlighted the importance of diagnosis and supervision in practical applications. In the evaluation of 1997, ISIS had recognised the importance of this topic and had announced its intention to develop a particular competence in this area. A plan to hire a new professor in this area had also been announced. This new position has not materialised, but it is very clear from the presentations we have heard that the Centre has developed a very important expertise in diagnosis and supervision applications. Five of the eleven ISIS projects have a substantial "diagnosis and supervision" component and, given the interest of the industrial partners for this topic, the Centre has decided to start a broad working group on the topic, in order to exchange the knowledge that has been gained so far.
International Collaboration and Ranking
We believe that one of the main reasons for the success of ISIS is the fact that the university groups have a world-wide reputation, which means that they are at the forefront of the scientific and technological progress through their many international contacts, visitors, post-doctoral fellows, etc. They undoubtedly rank among the leading groups in the world in their area of competence. Several of the industrial partners have stated that the main reason for their participation in ISIS is precisely the fact that the Linköping groups are known to be at the forefront of scientific knowledge.
Collaboration and Linkages within the Centre
A very strong point of ISIS is the beautiful collaboration between three groups from Electrical Engineering and three groups from Computer Science. We know from experience that such collaboration is not always easy to set up. This collaboration is particularly relevant given the range of topics covered by the project, which often require expertise in both fields. In our discussion with the PhD students, some of the Computer Science students also expressed their satisfaction at the close contacts with Electrical Engineering that they have through their training programme. This collaboration is at the heart of the existence and success of ISIS.
However, the progress in the integration of systems engineering and computer science research and projects has been hampered by a lot of instability within some of the Computer Science groups. We understand that a strong group on Data Base structures has left for Uppsala, and that the real-time and embedded systems group is in a reorganization phase as a result of the departure of their group leader to industry. The volatility in Computer Science departments is of course a well-known and world-wide phenomenon. However, we believe that the University administration and the ISIS Board should collaborate actively to quickly bring some new talent and stability to the Computer Science groups, so as to strengthen the Data-Base and Embedded Systems expertise within ISIS. The new strategic plan announced by the President of the University during our meeting, which provides for 4 new chairs in Computer Science and between 15 and 20 junior faculty positions, could go a long way to serve this objective. We strongly urge that two of these chairs, as well as some of the junior faculty positions, be invested in these areas of Computer Science that are central to the success of the ISIS programme.
As for the collaboration and linkages between the university groups and the industrial partners, these seem to have evolved in the best possible manner. The presence of all industrial companies on the Board, and the Technology Contact Groups (KPG) appear to play an important role here. We were told that these Technology Contact Groups meet on average about four times a year for technical discussions, but with varying degrees of success and participation, depending on the companies.
Identity and Management of the Centre
It is very apparent that the first five years of ISIS have resulted in a strong adhesion, by the whole team (both university and industrial partners), to the core values and goals of the Centre. These core values have been best summarised by the Director: "To find a working balance in such a way as to satisfy both the industrial needs and the achievement of scientific excellence". This adhesion, and the clear definition of the scientific/technical objectives, has given ISIS a well-defined identity. It is also obvious that the different categories of people in ISIS all express a great satisfaction at working together in this environment.
The management structure of ISIS contains the Board (made up of the industrial partners) which meets about 4 times a year, the Reference Group (made up of faculty members in charge of ISIS projects) which meets every month, and one Technology Contact Group (KPG) for each company. The members of the Reference Group are invited to attend the Board meetings. The addition of an industrial partner to ISIS requires unanimous approval by the Board.
In addition to their well-establhed participation in activities through the international technical community at large, the ISIS Centre has appointed a Scientific Advisory Board, which is utilised effectively to assess the programme and identify areas for improvement.
The Director is clearly the key driving force behind ISIS. His international reputation as a scientist and engineer, his natural leadership talents, and his very important investment in the activities and the management of the Centre make him the undisputed leader. He has obviously been able to create a team spirit that is rarely found in an organization that puts together people with such widely different cultures as university laboratories and competitive companies. He also clearly has open communication lines with the university structures, which were represented at our meeting by the Rector, the Vice-Rector, and the Dean.
The administration of ISIS is handled at a very low cost to NUTEK and to the apparent satisfaction of the participants. As one professor of Computer Science put it to us, one of the things he likes in ISIS is that he has so little administrative work, "thanks to the remarkable administrative efficiency of Lennart Ljung and Ulla Salaneck."
Finally, the structure and basic operation of ISIS is based on the following two strategic principles. First, the participating university units have additional research resources for more basic research work, which obviously benefits the ISIS projects. Second, projects are organized and selected consistently so as to be balanced "win-win" endeavours between the industry and university researchers. These strategies contribute in an essential way to the ISIS success as a Competence Centre. In this context, we were shown an example of a project that, while too theoretical to fit within ISIS, had been shifted to another source of funding because of its high fundamental relevance.
Conclusions and Recommendations
Since the last evaluation in 1997, the Competence Centre ISIS has clearly demonstrated its usefulness and efficiency. It clearly fills the role that NUTEK has assigned to it. It has made significant progress in establhing its cohesiveness and identity. All partners, university and industrial, express their satisfaction at the benefits they all get from ISIS, and at the growing integration of the themes and activities.
The major problem area in these last 3 years has been the volatility of qualified people on the Computer Science side. The university and the ISIS Board should pay special attention to this problem, by reinforcing the Computer Science groups through the hiring of one or more top quality persons. A search on the international scene should be encouraged.
3. Scientific and Technical Achievements
We have been told that, through the last three years, the Board of ISIS has consciously maintained the scope of its research programme in the five activity areas mentioned at the very beginning of this evaluation report. However, the projects run by ISIS have shown the particular importance, from an industrial application’s point of view, of the area of "diagnosis and detection". Clearly, this area has become a strong focal point of the research activities of the Centre. Given that this was perhaps not an area in which the university team leaders of ISIS had a world wide expertise, their approach has been to first do a thorough survey of the literature and of existing techniques in this area, and to then approach this field via a number of specialised applications. This has been highly successful, as we illustrate below, and in some cases this applied research has actually led to important new theoretical results.
The ISIS team is now at a turning point with respect to "diagnosis and detection". First, the expertise that they have built as a result of applications in this area should allow them to venture more solidly into theoretical contributions. Secondly, the broad interest across most industrial partners in this area has led them to decide to organize broad-based seminars and workshops on the subject.
Another area where the activity has been intense (and also spurred by some industrial applications) is data fusion. In this area also, there is clear cross-company interest, and the existence of ISIS therefore makes it possible for some companies to benefit from the know-how acquired on projects that were not their own.
We were very impressed by the fact that, already after a few years of operation, ISIS has produced several technical and commercially successful results. Let us mention the following successful transfers of technology:
Scientific Production and its Quality
The authors of the Centre report to the evaluation team have been very cautious (and honest) in listing only those scientific outputs that are directly the result of the ISIS Centre and funding. In its first four years of existence, the Centre has produced 7 PhD’s and 9 Licentiate Theses. To put these figures in perspective, one has to compare them with the total number of about 10 recurrent PhD students funded by ISIS. Thus, these figures indicate that the PhD’s that were started at the initiation of the ISIS programme have all been led to a successful end, and that the average duration of a PhD for an ISIS funded student is about 4 years. This was confirmed to us in the general discussion we have had with the students.
In this general discussion, the students expressed a great degree of satisfaction at doing their PhD in the framework of an ISIS project, because of the guaranteed relevance of their work. Some of them mentioned that there were occasional short term time constraints imposed upon them by the companies, but overall we had the feeling that, in the eyes of most students, the advantages of involvement in an industrial project far outweigh the occasional disadvantages due to deadlines. Clearly, the Director has also made a major effort to convince the companies that the PhD students must be given a lot of time to think and reflect, and that they should be as much as possible shielded from short term constraints.
As for the scientific publications, the report lists only those publications of which an ISIS funded student is a co-author. In the period 1996-99, the report lists 10 journal papers (of which 3 are bona fide publications in world class journals), 37 conference papers, and 4 patent applications. The number of bona fide journal papers may appear rather low, but one must take account of three factors: the long cycle from submission to publication, the fact that submitted or accepted papers are not listed in the report, and the fact that industrial projects do not lend themselves as easily to publications in the top quality journals.
It is clear that, through their more basic research funded outside the realm of ISIS, the university groups involved have a remarkably high scientific production, both in quality and quantity. We would nevertheless strongly encourage the ISIS group to attempt to publh a higher rate of journal papers as opposed to conference papers, given that journal papers have a much more lasting effect and reach a much broader audience.
Besides the traditional measure of quality based on publications and patents, we have, through our days of interviews and presentations, been able to convince ourselves of the high quality of the scientific work that is accomplhed within ISIS, particularly in the areas where there is a strong industrial motivation and relevance. In addition to the technical results mentioned in the previous subsection, let us mention the following high quality results, which have not yet led to industrial implementations.
We have some reservations about the project on "Design environment for real-time embedded systems in control-related applications". In our view, this project suffers from being too broad and overly ambitious. There is a risk that, by attempting to pose the design and supervision of control systems in such a broad way, the ISIS team may be lured too far away from its core competence. We comment more on this in the conclusions and recommendations below.
Education and Training
The collaboration between LiTH and the industrial companies within ISIS has very naturally meant that ISIS is an effective platform for these companies to recruit PhDs that they have had an opportunity to know and evaluate. What is perhaps more unusual is that ISIS has also spurred a movement from the companies to the universities: indeed, as a result of the collaboration, five engineers from the companies have joined the university research groups as PhD students.
The ISIS PhD students also benefit from the creation of a graduate school called ECSEL, funded by the Foundation for Strategic Research, SSF, that aims at providing a broader education across the fields of Computer Science and Systems Engineering.
Evidence was presented by ISIS personnel and by University officials substantiating the beneficial impact that ISIS has had within the University. From the ECSEL programme, to the innovative offerings of courses in one discipline targeted for students in complementary disciplines, to industry PhD students, ISIS has consistently led with pioneering introduction of a more cross-disciplinary nature.
Conclusions and Recommendations
It is clear that the first five years of the ISIS programme have shown the very high interest of the industrial partners for the fields of "diagnosis, detection and supervision", as well as "sensor fusion and data base management". These are two areas in which the joint efforts of the Electrical Engineering and Computer Science groups can bring about significant progress and contributions, insofar as they are closely related to the pre-existing areas of international competence of these groups, namely identification, estimation, non-linear estimation and control, analysis and verification of programs and systems.
The five activity areas of ISIS apparently cover a broader range of areas than those mentioned above. We believe that there is a potential risk at trying to cover too broad a field within the limited human resources that are available in ISIS. Indeed, the full accomplhment of the programme laid out in these 5 activity areas is extremely ambitious. We therefore recommend that, in the next phase, ISIS concentrates on the two areas mentioned above ("diagnosis, detection and supervision", and "data base management and sensor fusion") because they are closest to the disciplines in which they have strong and proven theoretical expertise.
In addition, we have observed that the projects that are performing best are those that have a strong component within these two areas, because they have a clearly defined industrial objective. These are also the two areas where there is a clear mutual interest that runs across companies.
4. Industrial Relevance and Benefits
Industrial Involvement and Commitment
A remarkable aspect of the participation of the industrial companies in ISIS, and a clear sign of their commitment, is that each of them has formulated its own strategy with respect to ISIS in writing. Excerpts of these statements can be found in the ISIS report prepared for this evaluation.
On the other hand, the Centre management has insisted that "projects must be in line with central, long-term activities in the companies". We were presented with evidence that this strategy is followed diligently. It is perhaps the strongest reason for the successful industry-university collaboration achieved in ISIS. The industrial interest is, as is indeed often the case, based on strong commitment of individual research managers at the industry. However, we found that commitments of this type were exceptionally strong within ISIS, which certainly is also an explanation for its success.
One of the apparent elements of success of ISIS is the line-up of the companies that are involved. They all have clear needs for a strong linkage with a group that has the outstanding scientific expertise that exists in the university groups, but at the same time they are not in competition with one another. This allows the scientific and technical discussions to be open and productive.
It is also interesting to observe that the success of an industrial collaboration between the control division of LiTH and Ericsson on power control in cellular radio systems has led Ericsson to join ISIS in the beginning of this year, with a new project on resource management in cellular phone systems.
Strengths in technology transfer and implementation of new technology
A very positive development of ISIS is the recent awareness, by the industrial partners, of the benefits of exchanging acquired knowledge across the projects. Whereas in the first few years of ISIS it appears that the collaboration was essentially between one company at the time (sometimes two) and the university groups, through a project, the industrial partners have now discovered that the ISIS projects have generated a generic know how in two or three areas that are of interest to most of them. These are the areas of sensor fusion, and of diagnosis and detection. As a result, ISIS is now about to start a new mechanism for the exchange of information and training in these areas through workshops and seminars.
The stable and long-term oriented relationship between ISIS and industry permits a coherent programme of MSc theses to be developed. Both university and industry benefit from this.
The emergence of two sub-areas of the ISIS research programme that attract interest from several companies, namely "diagnosis and detection" and "sensor fusion", is a very healthy element. We strongly encourage the university teams to organize workshops and seminars in these two areas. This will strengthen the identity and the cohesiveness of ISIS by creating a common culture, but it will also give the companies the opportunity to build up the expertise of their engineers in these areas that are bound to have a growing importance.
5. General Conclusions and Recommendations
It is our unanimous and strongly felt opinion that ISIS is a highly successful Competence Centre. It perfectly fulfils all the objectives of the NUTEK programme, with a particularly strong coherence in its objectives and commitment from its members. We therefore strongly recommend that NUTEK should continue to fund ISIS for the next period of the programme.
We are also satisfied with the way the management of ISIS has followed up on the recommendations of the earlier review of 1997.
The success encountered by the long-term university-industry collaboration establhed within ISIS shows that one can simultaneously pursue industrial relevance and scientific excellence, given the right conditions, funding and management. We believe that it is in the national interest of Sweden to think about the long term of such Centres, in order to give the Swedh high tech companies the competitive edge that they need.
In addition, the success of ISIS clearly has a very positive impact on the research productivity, the research training, and the graduate and undergraduate education at the University.
We therefore make the following specific recommendations to the partners of ISIS and, perhaps most importantly, to the University:
Linköping, September 25, 2000
Prof. John S. Baras Prof. Michel Gevers
Prof. Lino Guzzella Prof. Per Stenius