W4: Creative and Rigorous Model Development and Evaluation

Organised by Tony Jakeman and Teemu Kokkonen

Models are increasingly being relied upon to inform and support natural resource management. They are incorporating an ever broader range of disciplines and now often confront people without strong quantitative or model-building backgrounds. These trends imply a need for wider awareness of what constitutes good model-development practice, including reporting of models to users and sceptical review of models by users. To this end this workshop will consider as a starting point

The steps, which will be outlined in a draft Position Paper available soon, assume that modelling is intended to develop purposeful, credible models from data and prior knowledge, in consort with end-users, with every stage open to critical review and revision.

Best practice entails identifying clearly the clients and objectives of the modelling exercise; documenting the nature (quantity, quality, limitations) of the data used to construct and test the model; providing a strong rationale for the choice of model family and features (encompassing review of alternative approaches); justifying the techniques used to calibrate the model; serious analysis, testing and discussion of model performance; and making a resultant statement of model assumptions, utility, accuracy, limitations, and the need and potential for improvement.

Goals

Several authors and scientific communities have considered these issues previously. But we are attempting

The steps we delineate are appropriate whether the exercise employs: traditional models, e.g. the dynamic-statistical families of models, the empirical, deterministic or conceptual families; more recent artificial-intelligence-style or “knowledge-based” model types; other types: or a mixture.

The pursuit of good practice in model development and application deserves thorough and sustained attention, whatever the field. The aim of establishing good practice is

It assists in acceptance of this information and in long-term, systematic accrual of knowledge for science and decision-making.

This workshop

An example case study will be available soon as a guide for other contributors to the workshop. We are keen to attract other cases from a wide range of problems. Models involving the integration of several disciplines are especially welcome.

An outcome

The workshop will use these to discuss the constraints and ways forward

This template might be used to accompany journal articles published in the Environmental Modelling and Software journal. If we receive a sufficient number and variety of case studies we will publish them in a volume of the Elsevier IDEA book series on Developments in IntegrateD Environmental Assessment


Position Papers

  
Jakeman, A.J., Letcher, R.A, Norton, J.P. Ten Iterative Steps In Development and Evaluation of Environmental Models
Neil Crout, Glen Cox, James Gibbons Systematic Simplification of Mechanistic Models

Participants

  
Barbara Robson Ten Interactive Steps in Model Development Applied to Process-Based Modelling of Estuarine Biogeochemistry
James Gibbons, Glen Cox, Neil Crout, Andy Wood, Jim Craigon, Stephen Ramsden Applying Bayesian Model Averaging to Mechanistic Models
Glen Cox, James Gibbons, Jim Craigon, Stephen Ramsden, Andy Wood, Neil Crout Towards parsimony: generating alternative model formulations for assessment by model selection criteria.
Barry Croke Good Hydrological Modelling Practices in Data-poor Regions
Dean Holzworth The development of a farming systems model (APSIM)  a disciplined approach.
Andrew Gronewold, Robert Wolpert, Kenneth Reckhow Simple Models for Supporting Shellfish Resource Area Management Decisions
Knut Bernhardt Data-adaptive reduction of complex process-based models
Teemu Kokkonen, Harri Koivusalo, Tony Jakeman, John Norton Construction of a degree-day snow model in the light of the ten iterative steps in model development
Wendy Welsh Water Balance Modelling in Bowen, Queensland, and the Ten Iterative Steps In Model Development and Evaluation
Lachlan Newham, Tony Jakeman Ten interactive steps in model development: the construction, application and ongoing development of the CatchMODS water quality model
Sergei Schreider Four models for resource management: which mathematical method is more appropriate?
Valery Perminov On questions arising for any model simplification process
Yakov Pachepsky, Andrey Guber, Rien van Genuchten, Thomas Nicholson, Ralph Cady, Jirka Simunek, Timothy Gish, Diederik Jacques Model Abstraction In Hydrologic Modeling