W4: Creative and Rigorous Model Development and Evaluation

Organised by Tony Jakeman and Teemu Kokkonen

Models are increasingly being relied upon to inform and support natural resource management. They are incorporating an ever broader range of disciplines and now often confront people without strong quantitative or model-building backgrounds. These trends imply a need for wider awareness of what constitutes good model-development practice, including reporting of models to users and sceptical review of models by users. To this end this workshop will consider as a starting point

ten basic steps underpinning good or disciplined model practice.

The steps, which will be outlined in a draft Position Paper available soon, assume that modelling is intended to develop purposeful, credible models from data and prior knowledge, in consort with end-users, with every stage open to critical review and revision.

Best practice entails identifying clearly the clients and objectives of the modelling exercise; documenting the nature (quantity, quality, limitations) of the data used to construct and test the model; providing a strong rationale for the choice of model family and features (encompassing review of alternative approaches); justifying the techniques used to calibrate the model; serious analysis, testing and discussion of model performance; and making a resultant statement of model assumptions, utility, accuracy, limitations, and the need and potential for improvement.

Goals

Several authors and scientific communities have considered these issues previously. But we are attempting

to be more generic and to suggest guidelines for a wide range of model types.

The steps we delineate are appropriate whether the exercise employs: traditional models, e.g. the dynamic-statistical families of models, the empirical, deterministic or conceptual families; more recent artificial-intelligence-style or Òknowledge-basedÓ model types; other types: or a mixture.

The pursuit of good practice in model development and application deserves thorough and sustained attention, whatever the field. The aim of establishing good practice is

to enhance the credibility, and thus the impact, of the information and insight that modelling aims to generate.

It assists in acceptance of this information and in long-term, systematic accrual of knowledge for science and decision-making.

This workshop

invites case studies of model building that have been as thorough as possible in using best practice.

An example case study will be available soon as a guide for other contributors to the workshop. We are keen to attract other cases from a wide range of problems. Models involving the integration of several disciplines are especially welcome.

An outcome

The workshop will use these to discuss the constraints and ways forward

to design a recommended summary template for model reporting.

This template might be used to accompany journal articles published in the Environmental Modelling and Software journal. If we receive a sufficient number and variety of case studies we will publish them in a volume of the Elsevier IDEA book series on Developments in IntegrateD Environmental Assessment

Position Papers
Jakeman, A.J., Letcher, R.A, Norton, J.P.	Ten Iterative Steps In Development and Evaluation of Environmental Models
Neil Crout, Glen Cox, James Gibbons	Systematic Simplification of Mechanistic Models
Participants
Barbara Robson	Ten Interactive Steps in Model Development Applied to Process-Based Modelling of Estuarine Biogeochemistry
James Gibbons, Glen Cox, Neil Crout, Andy Wood, Jim Craigon, Stephen Ramsden	Applying Bayesian Model Averaging to Mechanistic Models
Glen Cox, James Gibbons, Jim Craigon, Stephen Ramsden, Andy Wood, Neil Crout	Towards parsimony: generating alternative model formulations for assessment by model selection criteria.
Barry Croke	Good Hydrological Modelling Practices in Data-poor Regions
Dean Holzworth	The development of a farming systems model (APSIM) a disciplined approach.
Andrew Gronewold, Robert Wolpert, Kenneth Reckhow	Simple Models for Supporting Shellfish Resource Area Management Decisions
Knut Bernhardt	Data-adaptive reduction of complex process-based models
Teemu Kokkonen, Harri Koivusalo, Tony Jakeman, John Norton	Construction of a degree-day snow model in the light of the ten iterative steps in model development
Wendy Welsh	Water Balance Modelling in Bowen, Queensland, and the Ten Iterative Steps In Model Development and Evaluation
Lachlan Newham, Tony Jakeman	Ten interactive steps in model development: the construction, application and ongoing development of the CatchMODS water quality model
Sergei Schreider	Four models for resource management: which mathematical method is more appropriate?
Valery Perminov	On questions arising for any model simplification process
Yakov Pachepsky, Andrey Guber, Rien van Genuchten, Thomas Nicholson, Ralph Cady, Jirka Simunek, Timothy Gish, Diederik Jacques	Model Abstraction In Hydrologic Modeling

W4: Creative and Rigorous Model Development and Evaluation

Position Papers

Participants