Viz Matters - New pages [en]

ReportAndRecommendations

2015-01-29T12:08:30Z

Mcassdtm:

=Summary list of Survey Executive Summaries:=
This pages give a summary of the conclusions of the global CCP survey (copied from the Main Page) plus summaries of the findings of the two smaller surveys of the CCPi and CFD communities. We also make recommendations for the future support of visualisation services and suggest how these surveys might be repeated in the future to monitor progress.

==For the Global CCP survey:==

*Three packages are the most-used packages by 26% of respondents. Conversely, another 31 packages are used by one or two users and account for a further 26% of respondents.
*Producing publication quality plots is the most-used technique.
*However, the features making these packages the favourites are:
#Software that is written specifically for their domain of interest.
#Large datasets are handled efficiently.
#Scripting or other ability to extend the tool is required.
*Users second most favoured packages are general purpose visualisation tools.
*Users were given five options for selecting their most required development. None emerged as being more needed than the others.
*Conversely, large amounts of memory was clearly the most important requirement for high performance visualisation.
*The main future challenges are suggested to be
#The ability to handle large amounts of data
#The ability to operate in a distributed environment.

==For the CCPi survey:==
The following conclusions were drawn:
*The survey was focused on what changes should be made to the set of supported tools? ImageJ, Avizo and ParaView products were favoured. A previous popular tool for development, VolView, proved to be unpopular with users and therefore was dropped.
*The type of software development supported should focus primarily on the areas where the network already has skills that can be leveraged (e.g. reconstruction) with new partners being sought to fill gaps (e.g. quantification and segmentation) for the future. Training was shown to be a requirement at different levels with outreach and metadata to be a lower priority.
*Over the following six months changes have happened that have resulted in the following actions:
#Quantification visualisation tools are now being developed with API links for ImageJ, Avizo and ParaView; the most popular specific tools.
#Outreach events are continuing but at a lower effort level.
#Software show-and-tell events along with related seminars now aim to focus on guidance and use of the products as well as the research solution that is being solved. These are now becoming monthly events and also aim to keep a user and developer community connected.
#Increase in cross training sessions: there is a collaboration now across Diamond Light Source / RCaH / University of Manchester and SCD/STFC labs with links to the Harwell Imaging Partnership and ISIS (Neutron Spallation Source). Four annual training events are being held at DLS/RCaH and a further four annual training events at UoM with specialist events being held at SCD/STFC in RAL - all of these allow for cross-attendance throughout the CCPi network.
#The survey allowed for representation at celebration events; including the Queen's Anniversary Prize at Manchester and the ToScA exhibition space where we sponsor poster prizes.
#Quarterly Developers' Workshop days are now held in CCP remote institutions and focus on areas that the survey stated there were needs.

==For the CFD survey:==
The survey of tools and techniques used within project gave the following main conclusions:
#Visualisation is most important in analysis; but also useful in problem definition, and mesh generation.
#ParaView is high on the list of preferred visualisation tools. But gnuplot, Matlab, xmgrace, and Pgplot are also required.
There are strong preferences to version number as features change. This may unify in the future.

=Recommendations for a Future Vis Service=

A future UK Vis-Service is worth exploring and a very worthy venture, but an obvious issue is the wide range and choice available of tools that any service would have to confront and support. Once all analysis is complete a list of discussion points that can be taken away will be placed here.

=WorkPackage Progress=

We were asked to do four workpackages that are being developed and presented throughout this site.

# WP1 We carried out an audit of the tools used and in a committee created a list of the common 34 packages for the range of CCPs. A further 30+ packages were mentioned in comments within the survey results. This in further work has been expanded to emphasise the long tail of tools available.
## 3D Slicer
## Amira
## atan
## [[Avizo]]
## Avogadro
## Chimera
## Coot/WinCoot
## DL Visualiser
## Drishti
## EnSight
## Fieldview
## Fiji
## [[gnuplot]]
## IDL
## ImageJ
## IMOD
## ITK/VTK
## Jmol
## [[MATLAB]]
## Octave
## OMERO
## OpenCV
## Paraview
## [[PyMol]]
## R
## RasMol
## SciLib
## tecplot
## VisIT
## VGStudio Max
## [[VMD]]
## VolView
## XCrySDen
## xmgrace
# WP2 User groups are very important as researchers and developers do not always associate CCP as their main motivation. The groups from the national facilities connected to STFC are included; DLS, ISIS and CLF. Other groups including the TSB (UK Innovate) Space Applications Catapult, RAL Space due to changing structure have been postponed for a future study. We also included CCPForge users directly: but only through news letters and a future study should directly address these users.
# WP3 The currently unfunded CCPs were invited to attend and augmented these values, as well as supported alternative bids.
# WP4 A final part in progress are certain software audits of other related organisations; including the German build IBM SuperMUC, two universities in Australia; visualisation suites at Curtin University and Australian National University and the Space applications Catapult.

=Future Roadmap=

There have been some requests for further work and also some groups delayed implementation. Currently there are four actions:

1. '''Two sub-surveys requested:''' We surveyed both the old non-funded CCPs which included

#[http://www.ccp.ac.uk/ CCP1] The electronic structure of molecules (defunct link)
#[http://www.ccp.ac.uk/ CCP3] Computational studies of surfaces (defunct link)
#[http://www.ccp.ac.uk/ CCP6] Molecular quantum dynamics
#[http://www.ccp.ac.uk/ CCP13] Software for fibre and polymer diffraction
#[http://www.ccp14.ac.uk/ CCP14] Powder diffraction

and surveyed the national facilities lists:

# [http://www.diamond.ac.uk/Home.html STFC: Diamond Light Source]
# [http://www.isis.stfc.ac.uk/ STFC: ISIS - Neutron Spallation Source]
# [http://www.stfc.ac.uk/clf/default.aspx STFC: CLF - Central Lasers Facility]

These links gathered 37 extra responses that were merged together with the total (6 from non-funded CCPs and 31 from the user communities). We have been asked to look specifically at the CLF and the ISIS user communities separately in a short extra report. From informal comments we believe users from non-funded CCPs have transferred to a funded CCP, a facilities user list, or moved on.

2. '''Rerunning the global survey:''' A further survey is being planned for Summer 2015, which will then track temporal changes between the CCPs and also indicate the changing nature from the new CCPs just recently funded.

3. '''Space Applications Catapult Audit:''' One part of the Optional Appendices was a page on the TSB Space Application Catapult Centre - just recently rebranded to becoming a UK Innovation site. They have recently built two new visualisation nodes and installed different and distinct software links to be reevaluated. This has come about due to change of use from a specific type of data show-and-tell space, to a command-and-control center with different visualisation needs. An associate is assisting in this move and the transition and thus as there is a new set of different users an audit is proposed in Spring 2015. We have requested this list to then compare against the Global CCP survey.

4. '''Global Comparison:''' We are in email contact with similar structures and have evaluated through the CCPi in particular specific tools from the Australian National University and Curtin University. Comparative study with audits are underway with these and at the German build IBM SuperMUC.

==Polishing Work==

We have started to look at a post-six month review to see where changes may have occurred; the first has complete with the CCPi initial survey and further work will be undertaken. We have been also asked to look at future user communities:

* CCP5 and CCP_EM as they embark on tomographic reconstruction and visualisation of 3D volumes in the next phase of CCP funding.
* CCP-NC involvement in remote visualisation and web based visualisation with Jmol for example. http://www.ccpnc.ac.uk/magresview/magresview/magres_view.html?JS
* CCP_PET/MR, a new recently funded CCP, that has specific 4D visualisation needs including the addition of uncertainty visualisation to human based 3D scans.

We are always open to taking on further studies and sub-studies, for other groups related to the CCP program.

==Authors==

Martin Turner [mailto:martin.turner@manchester.ac.uk email]

Ronald Fowler [mailto:ronald.fowler@stfc.ac.uk email]

Tim Morris [mailto:tim.morris@manchester.ac.uk email]

We welcome feedback and also thank all those that gave indirect and direct contributions.

CCPSurvey2014

2015-01-28T15:16:35Z

Zzalsmt2:

=Global Collaborative Computational Project (CCP) Survey results=

Over 2014 the visualisation group, part of the Technology Division within SCD, has reconsidered the real visualisation needs of the computational sciences community. A series of informal and formal surveys are underway and the first small ones have tackled the Tomographic Imaging and CFD (Computational Fluid Dynamics) communities. These wanted to find out which tools were actually being used and the best methods to support them. A global CCP survey was also undertaken and a discussion video conferencing presentation for this occured just before Christmas 2014.

[[File:Resbannerxmas_l.jpg|800px]]

== Executive Summary ==

Key was a difference between choice of primary and secondary package:

* Three packages are the most-used packages by 26% of respondents.
* Conversely, another 31 packages are used by one or two users and account for a further 26% of respondents.

Producing publication quality plots is the most-used technique. However, the features making these packages the favourites are:

# Software that is written specifically for their domain of interest.
# Large datasets are handled efficiently.
# Scripting or other ability to extend the tool is required.

Users second most favoured packages are general purpose visualisation tools.

Users were given five options for selecting their most required development. None emerged as being more needed than the others.

Conversely, large amounts of memory was clearly the most important requirement for high performance visualisation.

The main future challenges are suggested to be

# The ability to handle large amounts of data
# The ability to operate in a distributed environment.

== Survey Presentation ==

Surprisingly, open source was not always the most important issue but the easy creation of plug-ins, new readers and writers, as well as analysis tools have been requested.
There was also indicated a strong growth in the use of the ParaView visualisation system (http://www.paraview.org/) that is an open source, multi-platform data analysis and visualisation application where users can build systems including adding qualitative and quantitative techniques.

Q1 Home institutions

The wordle gives an indication of the distribution,

[[File:CCPSlide35.JPG|500px]]

but the institution links allows us to see the extremely long tail that makes the statistics not a power law relationship.

[[File:CCPSlide36.JPG|500px]]

Zooming in we can see the individual major establishments.

[[File:CCPSlide37.JPG|500px]]

Q2 Which CCP(s) are you involved with?

Looking at the 'old' CCPs there were very few responses which indicates users have moved on to new CCPs or institution facilities.

[[File:CCPSlide39.JPG|500px]]

The newer and therefore more current organisations have a larger response; with CCP5 and CCP9 being popular source code repositories and ISIS and CLF being popular user communities.

[[File:CCPSlide40.JPG|500px]]

Q3 What software do you use for visualisation of data?

Looking at the usage with both 'frequent' and 'essential' categories then we have the following curve.

[[File:CCPSlide41.JPG|500px]]

again we can zoom in and see in more detail the most popular tools.

[[File:CCPSlide42.JPG|500px]]

But removing the frequent and just looking at 'essential' tools the graph is very different and shows how different user communities use different tools.

[[File:CCPSlide43.JPG|500px]]

A wordle can also be useful to spot your favourite tools

[[File:CCPSlide44.JPG|500px]]

When asked about other tools there is an even longer tail to be found.

[[File:CCPSlide45.JPG|500px]]

The three most often used tools (gnuplot, MATLAB, Jmol) account for 26% of the responses and conversely, there is a tail of 31 packages that also accounts for 26% of the responses. Similarly the top four Essential tools (gnuplot, MATLAB, VMD, xmgrace) account for 42% of the responses. We need to ask why is there a long tail? Are there any useful features provided by the less popular tools that are not provided by gnuplot, MATLAB or Jmol? This is answered in the comments of questions 5 and 6.

Q4 What visualisation techniques are important to your work?

[[File:CCPSlide47.JPG|500px]]

This is less radical when we also add the 'occasional' used techniques.

[[File:CCPSlide48.JPG|500px]]

The most frequently used visualisation technique is to produce publication quality output. The facility to produce line graphs is equally important

Q5 Comments on the respondents' most used visualisation tool.

This section is converted into an interactive exploration. In summary 91 replies were given with the most commonly used tools, are:

* [[VMD]] - 9 replies
* [[MATLAB]] - 6 replies
* [[Avizo]] - 5 replies
* [[Gimias]] - 4 replies
* [[PyMol]] - 4 replies

Of the other tools, the major reasons for choosing them were:
* ASE: quick and easy to use
*Avogadro: open source
* gnuplot: quick to use, scriptable
* ImageJ: free, easy, plugins
* [[Paraview]]: easy of use

A few observations can be drawn from these responses;
# Users will prefer software that is written specifically for their domain of interest.
# Large datasets must be handled efficiently.
# Scripting or other ability to extend the tool is required.
# Publication quality output is a valued bonus

Q6 Same as question 5, for any other tool used.

Regarding the second most used visualisation tool 47 replies were given. The most commonly used tools, with links to answers are:
* [[gnuplot]] - 5 replies
* [[Materials Studio]] - 3 replies
* [[MATLAB]] - 3 replies, but is not discussed in this section as the responses are extremely similar to those in question 5.
* [[Paraview]] - 3 replies
* [[PyMol]] - 3 replies, also not discussed here.
* [[VMD]] - 3 replies, also not discussed here.

Other tools that had two responses:
* CCPN Analysis
* Discovery Studio
* IMOD
* JMOL
* Vesta
* Visit
* xmgrace

A few observations can be drawn from these responses:

# Users seems to prefer software that is general purpose.
# Large datasets must be handled efficiently.
# Good quality documentation/tutorials is required.
# Ability to read multiple formats is useful.

We can combine the frequency of tools in Q5 and Q6 to produce an interesting frequency graph.

[[File:CCPSlide54.JPG|500px]]

Q7 Visualisation requirements. How important do you see the provision of the following (ed. Services)?

[[File:CCPSlide55.JPG|500px]]

Q8 Requirements for high performance/advanced visualisation facilities. Do you have any need for access to (ed Services)

[[File:CCPSlide57.JPG|500px]]

Q9 What do you see as the main challenges for visualisation in your domain now and in the near future?

[[File:CCPSlide58.JPG|500px]]

Q10 Any other comments?

Summary
# Code development/maintenance in various guises
# Training in using various packages is required

Raw Comments
# Would be nice if there was an obvious preferred open source tomographic reconstruction code that we (STFC/EPSRC) could recommend to users and modify ourselves for our own needs. If there are candidates here, maybe publicise the options more widely?
# Would be nice to have somebody looking into maintenance and support of useful tools and software.
# Visualisation and application of calibrations to data go hand-in-hand together, thus the greatest visualisation tool won't be used much if it isn't integrated into a data handling flow. A good example is XCrysDen, which makes all the difference between just number crunching with wien2k and actually seeing what you're doing.
# It would be great if the visualisation tool developers would start to collaborate to develop libraries, data formats etc that could be shared between them, to speed up the development of new tools and functionality, and allow users access to the full spectrum of tools they require.
# I'm happy to get involved with this CCP/project or any initiative it might lead on with. I'm passionate about data visualisation and have experience in developing such tools.
# Training on visualisation tools are really lacking. There should be more of them.
# More seminars to introduce what is available
# There is a strong need for web-based visualization methods to integrate into web-based applications we are developing.
# License sharing and advanced usage training
# Lack of tools limits the science we are able to achieve.
# Final note - Martin Turner at RAL is brilliant at cross promotion (between communities) and compiling enthusiastic newsletters and should be thoughtfully praised for his dedication to the subject. He and his team are successfully promoting the ideas that other communities are way ahead in.
# I'm not sure if Diamond are as involved as they could be. If they are it certainly isn't communicated well to Users.
# Many of the packages are great for visualization but quick mathematics (e.g. subtract two 3d data sets) requires recoding.

== Future results ==

There will be a six month review in Spring 2015 - but remember from the presentation "The user and viewer are always important"

CFDSurvey

2015-01-28T14:59:57Z

Zzalsmt2:

=CFD Survey results=

This survey focussed on the Software for the future: Virtual Wave Structure Interaction Simulation Environment community.

[[File:LogoCFD.jpg|500px]]

The project was to implement highly parallel CFD solvers for nonlinear wave interactions aimed at marine wave structure interactions. There are collaboration between MMU, Plymouth and STFC, plus involvement with other projects (Froth). The main developments are within the OpenFOAM framework.

OpenFOAM;'s key visualisation platform is ParaView although with different cases using different versions. But ParaFoam for visualisation supports other formats including; Visit, Fluent, EnSight, TecPlot, FieldView, and VTK formats. The survey of visualisation requirements via Survey Monkey was still important to discuss alternative modules that could be developed for the framework.

[[File:AVF_SCD_CFD.jpg]]

== Executive Summary ==

From this survey the focus on choice of tools for the CFD SoFT project can be made. Survey of tools and techniques used within project gave the following main conclusions:

# Visualisation is most important in analysis; but also useful in problem definition, and mesh generation.
# Visualisation via ParaView is high on the list, but also the products gnuplot, Matlab, xmgrace, and Pgplot are required.

There are strong preferences to version number as features change. This may unify in the future.

== Presentation ==

A survey has been made of the visualisation methods that are currently employed by members of the SoFT project and their future needs.
The survey shows that ParaView is, as expected, the main tool for visualisation, as it comes built into OpenFOAM. It is also Open Source.
Gnuplot, pgplot and xmgrace are used for simple graphs. Matlab is the only commercial tool that was mentioned.

[[File:CFDSlide3.JPG|500px]]

The survey showed that 1D and 2D graphs are still considered important in understanding the details of CFD flow, along with publication quality output. High quality video and 3D views of the data are also important. Techniques such as line integral convolution and tensor plots are seen as less useful. Some respondents were particularly interested in the use of 3D stereo both in terms of being able to fly through the data and in ways to more easily present it to audiences without the need for expensive projection systems.

[[File:CFDSlide4.JPG|500px]]

Comments were asked for amd the following were given

[[File:CFDSlide5.JPG|500px]]

Extra points given included:

* Tecplot used by some for large data, 5GB to 1TB

* ParaView, EnSight, FieldView and VisIT also used

* Remote visualisation requested

* Stereo & immersive displays commented upon

== Results from Survey: November 2014 - April 2015 ==

This is under action and to be complete in Summer 2015.

CCPiSurvey

2015-01-28T12:08:16Z

Zzalsmt2:

=Collaborative Computational Project in Tomographic Imaging (CCPi) Survey results=

http://www.ccpi.ac.uk/

This survey was carried out during March-June 2014 to influence the panel during the Summer Working Group meeting. The key aim was to discuss how limited CCP core SLA effort could be better utilised by focussing on specific API development for certain platforms. Two secondary aims were first to consider which visualisation tools would be implementable within a new framework that was being proposed for the CCP, and the other secondary aim was to ask for feedback on the non-programming development work that the CCP was carrying out; for example training, advice etc.

[[File:CCPiWG.jpg]]

On 17 June 2014 the CCPi Working Group Meeting was held at the Atlas Visualisation Facility in RAL and the following presentation was given with feedback received to the CCP. Over the following six months to December 2014, progress has been focussed to address any issues and promote those areas that need enhancing.

== Executive Summary ==

From this survey the focus on choice of tools was changes so that ImageJ, Avizo and ParaView products were favoured. A previous popular tool for development VolView proved to be unpopular for users and therefore was dropped.

The type of software development supported should focus primarily on the areas where the network already has skills that can be leveraged (reconstruction) with new partners sought to fill gaps (quantification and segmentation) for the future. Training was shown to be a requirement at different levels with outreach and metadata to be a lower priority.

Over the following six months changes have happened that have resulted in the following actions:

* Quantification visualisation tools are now being developed with API links for ImageJ, Avizo and ParaView; the most popular specific tools.

* Outreach events are continuing but at a lower effort level.

* Software show-and-tell events along with related seminars now aim to focus on guidance and use of the products as well as the research solution that is being solved. These are now becoming monthly events and also aim to keep a user and developer community connected.

* Increase in cross training sessions: there is a collaboration now across Diamond Light Source / RCaH / University of Manchester and SCD/STFC labs with links to the Harwell Imaging Partnership and ISIS (Neutron Spallation Source). Four annual training events are being held at DLS/RCaH and a further four annual training events at UoM with specialist events being held at SCD/STFC in RAL - all of these allow for cross-attendance throughout the CCPi network.

* The survey allowed for representation at celebration events; including the Queen's Anniversary Prize at Manchester and the ToScA exhibition space where we sponsor poster prizes.

* Quarterly Developers' Workshop days are now held in CCP remote institutions and focus on areas that the survey stated there were needs.

This was not just a useful test but resulted in real actions that were guided by responses, and a further survey on CFD and then a global survey were carried out over the Summer and early Autumn of 2014.

== Presentation ==

Respondents came from a range of users through the 300+ names on the list but only 9 responded completely. The second question asked “What aspects of imaging are you involved with?” and gave a fixed set of responses.

[[File:CCPiSlide2.JPG|500px]]

This showed a predominance for user based needs rather than development needs; and also national facilities rather than lab based (often university type facilities) users.

Q3 "What software do you use for tomographic analysis and visualization?" With fixed list of tools.

[[File:CCPiSlide3.JPG|500px]]

This showed that there were a few key favoured packages that would be candidates for API links for future development.

Q4 "What image processing techniques do you use?" Choice of commercial/open source/own software.

[[File:CCPiSlide4.JPG|500px]]

This was interesting and significant, as it showed that users were developing their own software for reconstruction and some filtering techniques but mainly using commercial software for segmentation and quantification.

Q5: "What are your current and future computational requirements? Please detail your current needs in terms e.g. of image size and number of images. What computer hardware is typically required to process your data and is fast turn around important to your experiments?" Comments given were:

[[File:CCPiSlide5.JPG|500px]]

As expected scale and size of object was important although these values can be represented within large (1/2 TB RAM) fat nodes so extreme cluster implementations were not necessarily there to be developed.

Q6 "Are there any algorithms in tomographic image reconstruction, analysis or quantification that you believe CCPi should consider providing an efficient open source implementation of?" Comments given were:

[[File:CCPiSlide6.JPG|500px]]

This gave a starting point for a list of potential new objectives for the SLA team to address.

Q7 "CCPi is involved in a range of activities to support the research community (see http://www.cpi.ac.uk). Of the current CCPi activities which do you believe to be the most beneficial to the research community? Please rate each area on a scale of 1 to 5, with 5 being most important."

[[File:CCPiSlide7.JPG|500px]]

The obvious priority was training requirements - and discussion in the WG involved ways to set up cross training facilities and access to data.

Q8 "Do you have any other comments on the CCPi project?" Comments given were:

[[File:CCPiSlide8.JPG|500px]]

== Results from Survey: July-December 2014 ==

Over the following six months changes have happened that have resulted in the following actions, (with illustrations):

* Quantification visualisation tools are now being developed for [[ImageJ]], [[Avizo]] and [[Paraview]] the most popular specific tools.

* Outreach events are continuing but at a lower effort level. This includes the Eurographics EuroVis conference

[[File:OutreachEvent1.jpg|500px]]

and different KTN Materials exhibitions that had significant attendance.

[[File:OutreachEvent2.jpg|500px]]

* Software show-and-tell events along with related seminars focus on guidance and use of the products as well as the research solution. These are now monthly events to keep a user and developer community connected. Last 18 events have posters with images; so for 2013 - 2014 season there have been the following events

[[File:Semseason2013_2014.jpg|600px]]

and for 2014 - 2015 seasons we have so far had,

[[File:Semseason2014_2015.jpg|300px]]

* Increase in cross training sessions: there is now a collaboration across Diamond Light Source / RCaH / University of Manchester and SCD/STFC labs with links to the Harwell Imaging Partnership and ISIS. Four annual training events are held at DLS/RCaH and four annual training events are held at UoM and specialist events are held at SCD/STFC in RAL all allowing for cross-attendance throughout the CCPi network. Last two specialist events were on 8 September 2014 with Ajay Limaya (CCPi Short-term fellowship) on Drishti 2.5 with a morning training session and a afternoon data analysis hands on session.

[[File:ExtraTraining1.jpg|500px]]

and 8 May 2014 an Avizo 'experts' workshop was held at RAL considering use of v8.1

[[File:ExtraTraining2.jpg|500px]]

* The survey allowed for representation at celebration events; including the Queen's Anniversary Prize at Manchester and the ToScA exhibition space where we sponsor poster prizes.

[[File:QueensAward.jpg|500px]]

ToScA annual meetings:

[[File:SpecialEvent1.jpg|500px]]

[[File:SpecialEvent2.jpg|500px]]

* Quarterly Developers' Workshop days held in CCP remote institutions on various specific topics; Birmingham, Nottingham and QMUL.

[[File:Dwdays.jpg|500px]]

Materials Studio

2014-11-24T10:57:36Z

Mcassdtm: Created page with "http://accelrys.com/products/materials-studio/ Materials Studio "is a complete modeling and simulation environment designed to allow researchers in materials science and c..."

[[http://accelrys.com/products/materials-studio/ Materials Studio]] "is a complete modeling and simulation environment designed to allow researchers in materials science and chemistry to predict and understand the relationships of a material’s atomic and molecular structure with its properties and behavior. Using Materials Studio, researchers in many industries are engineering better performing materials of all types, including pharmaceuticals, catalysts, polymers and composites, metals and alloys, batteries and fuel cells, and more."

====Type of Data====
Chemical data, 3D volume, crystalline structure.

====Size of Data Set====
Megabytes of data or millions of 3D points.

====Advantages====
Advantages stem for it being a commercial package with substantial backing: is is well put together, fast, has high quality graphics and lots of features.

====Quality of Documentation/Tutorials====
Documentation is reported as being good or excellent.

====Range of Data Formats====
A good.excellent range of data formats is supported.

====Level of Support Available====
Good support is available - at a cost.

====Performance====
Reports of performance range for "OK" to "very good".

====Limitations====
*Expensive
*Only available in Windows
*Lacks the ability to interoperate with external software

====Comments====
Again, these comments reiterate earlier ones: the package is a commercial product.

Xmgrace

2014-11-21T16:52:46Z

Mcassdtm:

[[http://plasma-gate.weizmann.ac.il/Grace/ Xmgrace]] "is a WYSIWYG 2D plotting tool for the X Window System and M*tif. Grace runs on practically any version of Unix-like OS. As well, it has been successfully ported to VMS, OS/2, and Win9*/NT/2000/XP (some minor functionality may be missing, though)."

====Type of Data====

====Size of Data Set====

====Advantages====

====Quality of Documentation/Tutorials====

====Range of Data Formats====

====Level of Support Available====

====Performance====

====Limitations====

====Comments====

VESTA

2014-11-21T16:52:33Z

Mcassdtm:

===VESTA===
[[http://www.google.com VESTA]] ""

====Type of Data====

====Size of Data Set====

====Advantages====

====Quality of Documentation/Tutorials====

====Range of Data Formats====

====Level of Support Available====

====Performance====

====Limitations====

====Comments====

JMOL

2014-11-21T16:52:20Z

Mcassdtm:

[[http://jmol.sourceforge.net JMOL]] "is a free, open source molecule viewer for students, educators, and researchers in chemistry and biochemistry. It is cross-platform, running on Windows, Mac OS X, and Linux/Unix systems. "

====Type of Data====

====Size of Data Set====

====Advantages====

====Quality of Documentation/Tutorials====

====Range of Data Formats====

====Level of Support Available====

====Performance====

====Limitations====

====Comments====

Paraview

2014-11-21T16:52:07Z

Mcassdtm:

[[http://www.paraview.org ParaView]] "is an open-source, multi-platform data analysis and visualization application. ParaView users can quickly build visualizations to analyze their data using qualitative and quantitative techniques. The data exploration can be done interactively in 3D or programmatically using ParaView’s batch processing capabilities.

ParaView was developed to analyze extremely large datasets using distributed memory computing resources. It can be run on supercomputers to analyze datasets of petascale size as well as on laptops for smaller data, has become an integral tool in many national laboratories, universities and industry, and has won several awards related to high performance computation."

====Type of Data====
Will accept multiple data types

====Size of Data Set====
Respondents have processed data sets of up to a few TB

====Advantages====
*Good performance
*Free

====Quality of Documentation/Tutorials====
Good/Excellent documentation

====Range of Data Formats====
"the major formats can be read"

====Level of Support Available====
Good support is available.Tutorials and a user guide are available on the website. The developers also host a mailing list.

====Performance====
Performance is good, provided that sufficient memory is available for the data set - it can be run over multiple nodes to provide this.

====Limitations====
Remote connections with X system is slow.

====Comments====
The comments only reitereted points made previously: it is free and open source

Gnuplot

2014-11-21T16:51:54Z

Mcassdtm: /* Advantages */

[[http://gnuplot.info Gnuplot]] "is a portable command-line driven graphing utility for Linux, OS/2, MS Windows, OSX, VMS, and many other platforms. The source code is copyrighted but freely distributed (i.e., you don't have to pay for it). It was originally created to allow scientists and students to visualize mathematical functions and data interactively, but has grown to support many non-interactive uses such as web scripting. It is also used as a plotting engine by third-party applications like Octave. Gnuplot has been supported and under active development since 1986."

====Type of Data====
One respondent reported using all types of data with gnuplot.
Others were more selective.

====Size of Data Set====
Again, one respondent reported using any size of input. Another reported using 100s.

====Advantages====
Advantages are hinted at in the text quoted above, taken from gnuplot's home page: it is open source, flexible and mature.

====Quality of Documentation/Tutorials====
Documentation is reported to be good, although less so for advanced features. Because it is such a well-established tool, extensive online tutorial material is available.

====Range of Data Formats====
Huge flexibility, ranging from csv files to latex.

====Level of Support Available====
See above - extensive community support is available.

====Performance====
Is assessed as being "good".

====Limitations====
Input is by a scripting language that isn't to everyone's taste.

====Comments====
Comments reiterated statements made earlier, the package is open source and free.

PyMol

2014-11-21T10:39:20Z

Mcassdtm:

[[http://www.pymol.org PyMol]] "is an open-source, user-sponsored, molecular visualization system .... [it] can produce high-quality 3D images of small molecules and biological macromolecules, such as proteins. According to the original author, almost a quarter of all published images of 3D protein structures in the scientific literature were made using PyMOL."

====Type of Data====
Not surprisingly, given its intended audience, the users are generally viewing molecular structures. Two responses were more specific - protein structures.

====Size of Data Set====
Sizes of the data set were not specified.

====Advantages====
Advantages listed were:
*free version
*commonly used
*can create high quality output

====Quality of Documentation/Tutorials====
The documentation appears to be adequate. Although one user admitted to have not used it.

====Range of Data Formats====

====Level of Support Available====
Support was acceptable.

====Performance====
None of the respondents commented on performance.

====Limitations====
One user commented that it could be slow for large data sets.

====Comments====
No further comments were given.

Gimias

2014-11-21T10:38:58Z

Mcassdtm:

[[http://www.gimias.org Gimias]] "is a workflow-oriented environment for solving advanced biomedical image computing and individualized simulation problems, which is extensible through the development of problem-specific plug-ins. In addition, GIMIAS provides an open source framework for efficient development of research and clinical software prototypes integrating contributions from the Physiome community while allowing business-friendly technology transfer and commercial product development."

====Type of Data====
Gimias is used mostly to process mesh data, but also 2 and 3D scalar images, and, more generally, medical images.

====Size of Data Set====
Data sets are:
*"millions of elements"
*10kB to 15GB
*up to 1GB
*few MB to GB

====Advantages====

*Gimias allows rapid prototyping of clinical applications.
*It is open-source
*Cross-platform

====Quality of Documentation/Tutorials====
Documentation is applauded. "not extensive, but useful", "excellent". It appears to cater for the whole range of users - novice to expert.

====Range of Data Formats====
"most of them" is a phrase used. In addition, it provides the facility to create a reader for any new format

====Level of Support Available====
Respondents state it has a very responsive development team.

====Performance====
Performance is uniformly described as good.

====Limitations====
It seems to have no limitations.

====Comments====
The one comment, made in 3 of the 4 responses is that this is open source , BSD licence, and extensible.

Avizo

2014-11-21T10:38:38Z

Mcassdtm:

[[http://www.fei.com/software/avizo3d/ Avizo]] "is a 3D analysis software for scientific and industrial data. Wherever three-dimensional imaging data sets need to be processed, in materials science, geosciences or engineering applications, Avizo offers abundant state-of-the-art image data processing, exploration and analysis features within an intuitive workflow and easy-to-use graphical user interface."

====Type of Data====
Avizo is designed for analysing and manipulating 3D data sets. It is therefore no surprise that the data sets used by respondents were exclusively tomographic datasets.

====Size of Data Set====
Data sets were typically 20GB, with one respondent having much smaller data sets and one significantly larger (100GB).

====Advantages====
The only advantage stated was that Avizo can generate publication quality images.

====Quality of Documentation/Tutorials====
Documentation is OK, but additional tutorials (on YouTube) would be useful.

====Range of Data Formats====
The range of data formats accepted is acceptible.

====Level of Support Available====
Two users suggested the support ;eve; was good, two said not.

====Performance====
Performance is probably dictated by the volume of data that is processed. A High end workstation is required.

====Limitations====
More limitations were listed for this package than any other:
*graphics card crashes
*Avizo crashes all the time
*Minimal documentation
*Can be too complicated to use

====Comments====
A further limitation was given in the comments:
*it is commercial and expensively so

MATLAB

2014-11-21T10:38:12Z

Mcassdtm:

[[http://uk.mathworks.com/products/matlab/?nocookie=true MATLAB]] "is a high-level language and interactive environment for numerical computation, visualization, and programming. Using MATLAB, you can analyze data, develop algorithms, and create models and applications. The language, tools, and built-in math functions enable you to explore multiple approaches and reach a solution faster than with spreadsheets or traditional programming languages, such as C/C++ or Java™."

====Type of Data====
Matlab appears to cope with the widest range of numerical data: CSV files, time series, surfaces, lines in 3D space were mentioned. Input can be summarised as "numerical arrays/data"

====Size of Data Set====
A wide range of sizes of arrays were used, ranging from 10kB up to tens of GB

====Advantages====
Advantages mentioned are those that accrue from a mature, widely used and respected tool:
*ease of use
*scripting
*known and used worldwide
*excellent library
*integration with Python

====Quality of Documentation/Tutorials====
Was uniformly regarded as good.

====Range of Data Formats====
Also uniformly regarded as good.

====Level of Support Available====
Responses were more variable, spanning the opinions from very good to average.

====Performance====
It is widely supposed that Matlab's performance is poor since it is interpreted at runtime. In practice, the functions are so well optimised that performance is surprisingly good, but cannot approach that of a compiled programme. One problem nobody mentioned in the survey, but has been encountered by us, is that Matlab acquires licences for its toolboxes at runtime, in a large organisation these can be limited, and this can result in the failure of a function to execute.

====Limitations====
Two limitations were mentioned:
*it is limited to a personal computer
*the tutorials can be technical

====Comments====
Only one other comment was made: Matlab is a commercial product.

MATLB

2014-11-21T10:37:31Z

Mcassdtm: Created page with "====MATLAB===="

====MATLAB====

VMD

2014-11-21T10:37:01Z

Mcassdtm:

[[http://www.ks.uiuc.edu/Research/vmd/ VMD]] "is designed for modeling, visualization, and analysis of biological systems such as proteins, nucleic acids, lipid bilayer assemblies, etc. It may be used to view more general molecules, as VMD can read standard Protein Data Bank (PDB) files and display the contained structure. VMD provides a wide variety of methods for rendering and coloring a molecule: simple points and lines, CPK spheres and cylinders, licorice bonds, backbone tubes and ribbons, cartoon drawings, and others. VMD can be used to animate and analyze the trajectory of a molecular dynamics (MD) simulation. In particular, VMD can act as a graphical front end for an external MD program by displaying and animating a molecule undergoing simulation on a remote computer."

====Type of Data====

Unsurprisingly, most respondents use this software for molecular/atomic/protein visualisation.

====Size of Data Set====

This varies widely. About half the respondents have data sets smaller than a few GB. Two had data sets of 40 and 50 GB. Otherwise they had 1,000 or 30,000 atoms.

====Advantages====

The most widely cited advantages are
*quality of output
*range of supported file types
*simple to use/script

====Quality of Documentation/Tutorials====

Documentation was almost universally regarded as good or very good. One respondent stated it was difficult to navigate.

====Range of Data Formats====

Again, almost universally regarded as good, with two exceptions:
*"temperamental with periodic images"
*"no support for cif"

====Level of Support Available====

A range of resources is used: mailing lists and online forum.

====Performance====

Performance generally regarded as good, especially on a desktop machine.

====Limitations====

*"Needs a good graphics card"
*"TCL scripting is limited"
*"Slow to load large data sets"
*"Some additional analysis tools are unstable"
*"Some representations not appropriate for periodic boundary conditions"
*"Cannot interpret bond types"

====Comments====

The tools are free and open source

Q10

2014-11-05T11:27:53Z

Mcassdtm: /* Summary */

===Any other comments?===

====Raw Comments====
#Would be nice if there was an obvious preferred open source tomographic reconstruction code that we (STFC/EPSRC) could recommend to users and modify ourselves for our own needs. If there are candidates here, maybe publicise the options more widely?
#Would be nice to have somebody looking into maintenance and support of useful tools and software.
#Visualisation and application of calibrations to data go hand-in-hand together, thus the greatest visualisation tool won't be used much if it isn't integrated into a data handling flow. A good example is XCrysDen, which makes all the difference between just number crunching with wien2k and actually seeing what you're doing.
#It would be great if the visulisation tool developers would start to collaborate to develop libraries, data formats etc that could be shared between them, to speed up the development of new tools and functionality, and allow users access to the full spectrum of tools they require.
#I'm happy to get involved with this CCP/project or any initiative it might lead on with. I'm passionate about data visualisation and have experience in developing such tools.
#Training on visualisation tools are really lacking. There should be more of them.
#More seminars to introduce what is available
#There is a strong need for web-based visualization methods to integrate into web-based applications we are developing.
#License sharing and advanced usage training
#Lack of tools limits the science we are able to achieve.
#Final note - Martin Turner at RAL is brilliant at cross promotion (between communities) and compiling enthusiastic newsletters and should be thoughtfully priased for his dedication to the subject. He and his team are successfully promoting they ideas that other communities are way ahead in.
#I'm not sure if Diamond are as involved as they could be. If they are it certainly isn't communicated well to Users.
#Many of the packages are great for visualization but quick mathematics (e.g. subtract two 3d data sets) requires recoding.

====Summary====
#Code development/maintenance in various guises
#Training in using various packages is required

Q9

2014-11-05T11:27:32Z

Mcassdtm:

===What do you see as the main challenges for visualisation in your domain now and in the near future? ===

The respondents' were given the opportunity to make comments, with no prompting. Clustering of the responses gave this group of clusters

[[File:vissurveycompen1.jpg]]

====Summary====
*The size of data sets to be visualised is the most pressing challenge. The can be exposed in several ways:
#how can large data sets be visualised in real-time?
#facilities are limited for long term storage of large data sets.
#time required to transfer data between facilities for remote viewing (or distributed processing) is a limiting factor to its usefulness.
#large volumes of data require a quick method for generating an overview, before performing a closer inspection.
#reading the data is time consuming.

*Remote visualisation
#Especially a problem using very large scale supercomputing resources overseas
#Client-server rendering is slow
#Equally important is moving data from data capture to data processing locations

*Input and Output
#As stated above, loading large data sets is the bottleneck

*Quantification
#Visualisation is limited in scope becasue research outputs require quantifiable data

*3D/Immersive

*User Experience
#No package provides all of the required functionality (was mentioned multiple times)
#Absence of expertise amongst users

*Computational Time/Parallelism
#As mentioned above, there is a requirement to visualise data as it is being captured.

Q8

2014-11-05T11:27:13Z

Mcassdtm:

===Requirements for high performance/advanced visualisation facilities. Do you have any need for access to: ===

The question asked about facilities in use or required. The facilities asked about were:

*Large memory on single workstation (>64Gbytes)
*Large memory or compute visualisation using computer clusters
*Very high resolution displays
*3D stereo display (individual user)
*3D stereo display (room sized)
*Head tracking/immersive stereo
*Haptic feedback devices
*Other (please specify)

No "Others" were specified in the responses.

The following chart shows the proportions of users already using the facilities (n = 77):

[[File:Q8alreadyuse.png]]

It is apparent that about one third of respondents already use workstations with large memories, and one third also have access to computer clusters with large memories. 3D stereo is also quite widely used. Nobody used haptic feedback.

The following pie chart shows the proportions of each facility on the wish list (n = 189):

[[File:Q8wouldlike.png]]

The most requested facilities are large amounts of memory (workstation or cluster) at 20% each, and very high resolution displays at 21% of respondents. 3D stereo on a workstation is also quite often requested. Haptic feedback was requested by 5% of respondents.

Q7

2014-11-05T11:26:42Z

Mcassdtm:

===Visualisation requirements. How important do you see the provision of the following? ===

The question gave five requirements, respondents were asked to given them a priority on a scale of 1 to 5.

The requirements were:

*Courses on visualisation techniques

*Courses on using common tools

*Plugin developments

*Development of new visualisation tools

*Documentation of visualisation tools

The pie chart shows the average response per question, i.e. the bigger the sector, the more priority was assigned to this option by respondents (n = 262).

[[File:Q7.png]]

Whilst there is no clearly preferred requirement, "Documentation of tools" was most requested. "Development of new ''plugins''" was the second most requested requirement, but "Development of new ''tools''" was least requested. But, the differences between the score are probably not significant.

In addition, the opportunity for comments was given, these seem to have been used to justify or amplify respondents' answers:

#visualisation is usually specialised for the data type/experiment. With hybrid approaches, integration of different data types becomes critical.
#I am not sure if you mean hardware (as in large screens) or hardware (as in memory, graphics and processors). The large screens are of no use to me. It's all about easy and reliable access to stable, non-frustrating platforms. As long as my screen is big, I do NOT need it huge! I just want stable.
#Performance when working over a network
#Examples: Many of the 3d stuff lacks examples!! Complete downloadable example, not fragments.
#My first thought is that individual tools shouldn't need courses, but if well designed their use should be self explanatory even without reading documentation. Courses on Visualisation techniques might be better, but even then it should be enough to see a neat visualisation and then have it be relatively obvious how that might be achieved with a powerful enough general purpose tool.
#Visualization of time varying data. Information visualization. Big data.
#User interface to generate a script for automatisation
#On the fly/parallel visualisation, ideally remotely to/from large repositories/archives/curation centres. Better tools for immersive 3D.
#The development of a new visualisation tool could only be justified if it did something really different from, or a lot better than, that which already exists

Q6

2014-11-05T11:26:16Z

Mcassdtm:

===Comments on the respondents' second most used visualisation tool. ===

47 replies were given.

The most commonly used tools, with links to answers are:
#[[gnuplot]] - 5 replies
#[[Materials Studio]] - 3 replies
#Matlab - 3 replies, but is not discussed in this section as the responses are the same as those in question 5.
#[[Paraview]] - 3 replies
#PyMol - 3 replies, also not discussed here.
#VMD - 3 replies, also not discussed here.

Other tools that had two responses:
#CCPN Analysis
#Discovery Studio
#IMOD
#JMOL
#Vesta
#Visit
#xmgrace

====Overall====
A few observations can be drawn from these responses:
#Users seems to prefer software that is general purpose.
#Large datasets must be handled efficiently.
#Good quality documentation/tutorials is required.
#Ability to read multiple formats is useful.

Q5

2014-11-05T11:25:55Z

Mcassdtm: /* Comments on the respondents' most used visualisation tool. */

===Comments on the respondents' most used visualisation tool. ===

91 replies were given.

The most commonly used tools, with links to answers are:
#[[VMD]] - 9 replies
#[[MATLAB]] - 6 replies
#[[Avizo]] - 5 replies
#[[Gimias]] - 4 replies
#[[PyMol]] - 4 replies

Of the other tools, the major reasons for choosing them were:
*ASE: quick and easy to use
*Avogadro: open source
*gnuplot: quick to use, scriptable
*ImageJ: free, easy, plugins
*Paraview: ease of use

====Overall====
A few observations can be drawn from these responses:
#Users will prefer software that is written specifically for their domain of interest.
#Large datasets must be handled efficiently.
#Scripting or other ability to extend the tool is required.
#Publication quality output is a valued bonus.

Q4

2014-11-05T11:25:30Z

Mcassdtm:

===What visualisation techniques are important to your work?===

We provided a list of what we believed to be the important techniques used by members of the CCPs. The question asked about how often these were used. Answers were one of: not needed, may be useful or frequently used. The histogram represents the results ordered by the summation of the positive replies.

[[File:Q4.png]]

====Observations====
*Looking at the frequently used techniques:
#The most frequently used visualisation technique is to produce publication quality output.
#The facility to produce line graphs is equally important
*Looking at the combined "frequently used" and "may be useful" techniques
#There is a less clear distinction between the top techniques:
#Publication quality output is the most used
#Simplified visualisation methods are also required

Q3

2014-11-05T11:25:10Z

Zzalsmt2:

===What software do you use for visualisation of data? ===

We provided a list of packages, responses were one of: not used, occasionally used, frequently used or essential.

Note that we've combined the numbers of people who use each tool, the frequency of use is colour coded.

[[File:Q3a.png]]

====Observations====
#The three most frequently used tools (gnuplot, MATLAB, Jmol) account for 26% of the responses.
#Conversely, there is a tail of 31 packages that also accounts for 26% of the responses.
#The top four Essential tools (gnuplot, MATLAB, VMD, xmgrace) account for 42% of the responses.

We need to ask why is there a long tail? Are there any useful features provided by the less popular tools that are not provided by gnuplot, MATLAB or Jmol? This is answered in the comments of questions 5 and 6.

Respondents were given the opportunity to mention any tools not listed:

[[File:Q3b.png]]

41 tools are mentioned, used by a total of 76 respondents. It is again interesting to discover why these tools are used. We attempt to answer this with the comments of questions 5 and 6.

Other new tools listed are:
# plotly https://plot.ly/

Q2

2014-11-05T11:21:25Z

Mcassdtm: /* Which CCP(s) are you involved with? */

===Which CCP(s) are you involved with?===

This is the histogram showing participants' involvement with the CCPs. Some respondent's were involved with multiple CCPs.

[[File:Q2.png]]