Viz Matters - User contributions [en]

ReportAndRecommendations

2015-01-30T13:12:59Z

Mcassdtm:

=Summary list of Survey Executive Summaries:=
This pages give a summary of the conclusions of the global CCP survey (copied from the Main Page) plus summaries of the findings of the two smaller surveys of the CCPi and CFD communities. We also make recommendations for the future support of visualisation services and suggest how these surveys might be repeated in the future to monitor progress.

==For the Global CCP survey:==

*Three packages are the most-used packages by 26% of respondents. Conversely, another 31 packages are used by one or two users and account for a further 26% of respondents.
*Producing publication quality plots is the most-used technique.
*However, the features making these packages the favourites are:
#Software that is written specifically for their domain of interest.
#Large datasets are handled efficiently.
#Scripting or other ability to extend the tool is required.
*Users second most favoured packages are general purpose visualisation tools.
*Users were given five options for selecting their most required development. None emerged as being more needed than the others.
*Conversely, large amounts of memory was clearly the most important requirement for high performance visualisation.
*The main future challenges are suggested to be
#The ability to handle large amounts of data
#The ability to operate in a distributed environment.

==For the CCPi survey:==
The following conclusions were drawn:
*The survey was focused on what changes should be made to the set of supported tools? ImageJ, Avizo and ParaView products were favoured. A previous popular tool for development, VolView, proved to be unpopular with users and therefore was dropped.
*The type of software development supported should focus primarily on the areas where the network already has skills that can be leveraged (e.g. reconstruction) with new partners being sought to fill gaps (e.g. quantification and segmentation) for the future. Training was shown to be a requirement at different levels with outreach and metadata to be a lower priority.
*Over the following six months changes have happened that have resulted in the following actions:
#Quantification visualisation tools are now being developed with API links for ImageJ, Avizo and ParaView; the most popular specific tools.
#Outreach events are continuing but at a lower effort level.
#Software show-and-tell events along with related seminars now aim to focus on guidance and use of the products as well as the research solution that is being solved. These are now becoming monthly events and also aim to keep a user and developer community connected.
#Increase in cross training sessions: there is a collaboration now across Diamond Light Source / RCaH / University of Manchester and SCD/STFC labs with links to the Harwell Imaging Partnership and ISIS (Neutron Spallation Source). Four annual training events are being held at DLS/RCaH and a further four annual training events at UoM with specialist events being held at SCD/STFC in RAL - all of these allow for cross-attendance throughout the CCPi network.
#The survey allowed for representation at celebration events; including the Queen's Anniversary Prize at Manchester and the ToScA exhibition space where we sponsor poster prizes.
#Quarterly Developers' Workshop days are now held in CCP remote institutions and focus on areas that the survey stated there were needs.

==For the CFD survey:==
The survey of tools and techniques used within project gave the following main conclusions:
#Visualisation is most important in analysis; but also useful in problem definition, and mesh generation.
#ParaView is high on the list of preferred visualisation tools. But gnuplot, Matlab, xmgrace, and Pgplot are also required.
There are strong preferences to version number as features change. This may unify in the future.

=Recommendations for a Future Vis Service=

A future UK Vis-Service is worth exploring and a very worthy venture, but an obvious issue is the wide range and choice available of tools that any service would have to confront and support. Once all analysis is complete a list of discussion points that can be taken away will be placed here.

=WorkPackage Progress=

We were asked to do four workpackages that are being developed and presented throughout this site.

# WP1 We carried out an audit of the tools used and in a committee created a list of the common 34 packages for the range of CCPs. A further 30+ packages were mentioned in comments within the survey results. This in further work has been expanded to emphasise the long tail of tools available.
## 3D Slicer
## Amira
## atan
## [[Avizo]]
## Avogadro
## Chimera
## Coot/WinCoot
## DL Visualiser
## Drishti
## EnSight
## Fieldview
## Fiji
## [[gnuplot]]
## IDL
## ImageJ
## IMOD
## ITK/VTK
## Jmol
## [[MATLAB]]
## Octave
## OMERO
## OpenCV
## Paraview
## [[PyMol]]
## R
## RasMol
## SciLib
## tecplot
## VisIT
## VGStudio Max
## [[VMD]]
## VolView
## XCrySDen
## xmgrace
# WP2 User groups are very important as researchers and developers do not always associate CCP as their main motivation. The groups from the national facilities connected to STFC are included; DLS, ISIS and CLF. Other groups including the TSB (UK Innovate) Space Applications Catapult, RAL Space due to changing structure have been postponed for a future study. We also included CCPForge users directly: but only through news letters and a future study should directly address these users.
# WP3 The currently unfunded CCPs were invited to attend and augmented these values, as well as supported alternative bids.
# WP4 A final part in progress are certain software audits of other related organisations; including the German build IBM SuperMUC, two universities in Australia; visualisation suites at Curtin University and Australian National University and the Space applications Catapult.

=Future Roadmap=

There have been some requests for further work and also some groups delayed implementation. Currently there are four actions:

1. '''Two sub-surveys requested:''' We surveyed both the old non-funded CCPs which included

#[http://www.ccp.ac.uk/ CCP1] The electronic structure of molecules (defunct link)
#[http://www.ccp.ac.uk/ CCP3] Computational studies of surfaces (defunct link)
#[http://www.ccp.ac.uk/ CCP6] Molecular quantum dynamics
#[http://www.ccp.ac.uk/ CCP13] Software for fibre and polymer diffraction
#[http://www.ccp14.ac.uk/ CCP14] Powder diffraction

and surveyed the national facilities lists:

# [http://www.diamond.ac.uk/Home.html STFC: Diamond Light Source]
# [http://www.isis.stfc.ac.uk/ STFC: ISIS - Neutron Spallation Source]
# [http://www.stfc.ac.uk/clf/default.aspx STFC: CLF - Central Lasers Facility]

These links gathered 37 extra responses that were merged together with the total (6 from non-funded CCPs and 31 from the user communities). We have been asked to look specifically at the CLF and the ISIS user communities separately in a short extra report. From informal comments we believe users from non-funded CCPs have transferred to a funded CCP, a facilities user list, or moved on.

2. '''Rerunning the global survey:''' A further survey is being planned for Summer 2015, which will then track temporal changes between the CCPs and also indicate the changing nature from the new CCPs just recently funded.

3. '''Space Applications Catapult Audit:''' One part of the Optional Appendices was a page on the TSB Space Application Catapult Centre - just recently rebranded to becoming a UK Innovation site. They have recently built two new visualisation nodes and installed different and distinct software links to be reevaluated. This has come about due to change of use from a specific type of data show-and-tell space, to a command-and-control center with different visualisation needs. An associate is assisting in this move and the transition and thus as there is a new set of different users an audit is proposed in Spring 2015. We have requested this list to then compare against the Global CCP survey.

4. '''Global Comparison:''' We are in email contact with similar structures and have evaluated through the CCPi in particular specific tools from the Australian National University and Curtin University. Comparative study with audits are underway with these and at the German build IBM SuperMUC.

==Polishing Work==

We have started to look at a post-six month review to see where changes may have occurred; the first has complete with the CCPi initial survey and further work will be undertaken. We have been also asked to look at future user communities:

* CCP5 and CCP_EM as they embark on tomographic reconstruction and visualisation of 3D volumes in the next phase of CCP funding.
* CCP-NC involvement in remote visualisation and web based visualisation with Jmol for example. http://www.ccpnc.ac.uk/magresview/magresview/magres_view.html?JS
* CCP_PET/MR, a new recently funded CCP, that has specific 4D visualisation needs including the addition of uncertainty visualisation to human based 3D scans.

We are always open to taking on further studies and sub-studies, for other groups related to the CCP program.

==Authors==

Martin Turner [mailto:martin.turner@manchester.ac.uk email]

Ronald Fowler [mailto:ronald.fowler@stfc.ac.uk email]

Tim Morris [mailto:tim.morris@manchester.ac.uk email]

We welcome feedback and also thank all those that gave indirect and direct contributions.

Mcassdtm: /* Presentation */

Collaborative Computational Project in Tomographic Imaging (CCPi) Survey results.

http://www.ccpi.ac.uk/

This survey was carried out during March-June 2014 to influence the panel during the Summer Working Group meeting. The key aim was to discuss how limited CCP core SLA effort could be better utilised by focussing on specific API development for certain platforms. Two secondary aims were first to consider which visualisation tools would be implementable within a new framework that was being proposed for the CCP, and the other secondary aim was to ask for feedback on the non-programming development work that the CCP was carrying out; for example training, advice etc.

[[File:CCPiWG.jpg]]

On 17 June 2014 the CCPi Working Group Meeting was held at the Atlas Visualisation Facility in RAL and the following presentation was given with feedback received to the CCP. Over the following six months to December 2014, progress has been focussed to address any issues and promote those areas that need enhancing.

== Executive Summary ==

From this survey the focus on choice of tools was changes so that ImageJ, Avizo and ParaView products were favoured. A previous popular tool for development VolView proved to be unpopular for users and therefore was dropped.

The type of software development supported should focus primarily on the areas where the network already has skills that can be leveraged (reconstruction) with new partners sought to fill gaps (quantification and segmentation) for the future. Training was shown to be a requirement at different levels with outreach and metadata to be a lower priority.

Over the following six months changes have happened that have resulted in the following actions:

* Quantification visualisation tools are now being developed with API links for ImageJ, Avizo and ParaView; the most popular specific tools.

* Outreach events are continuing but at a lower effort level.

* Software show-and-tell events along with related seminars now aim to focus on guidance and use of the products as well as the research solution that is being solved. These are now becoming monthly events and also aim to keep a user and developer community connected.

* Increase in cross training sessions: there is a collaboration now across Diamond Light Source / RCaH / University of Manchester and SCD/STFC labs with links to the Harwell Imaging Partnership and ISIS (Neutron Spallation Source). Four annual training events are being held at DLS/RCaH and a further four annual training events at UoM with specialist events being held at SCD/STFC in RAL - all of these allow for cross-attendance throughout the CCPi network.

* The survey allowed for representation at celebration events; including the Queen's Anniversary Prize at Manchester and the ToScA exhibition space where we sponsor poster prizes.

* Quarterly Developers' Workshop days are now held in CCP remote institutions and focus on areas that the survey stated there were needs.

This was not just a useful test but resulted in real actions that were guided by responses, and a further survey on CFD and then a global survey were carried out over the Summer and early Autumn of 2014.

== Presentation ==

Respondents came from a range of users through the 300+ names on the list but only 9 responded completely. The second question asked “What aspects of imaging are you involved with?” and gave a fixed set of responses.

[[File:CCPiSlide2.JPG|500px]]

This showed a predominance for user based needs rather than development needs; and also national facilities rather than lab based (often university type facilities) users.

Q3 "What software do you use for tomographic analysis and visualization?" With fixed list of tools.

[[File:CCPiSlide3.JPG|500px]]

This showed that there were a few key favoured packages that would be candidates for API links for future development.

Q4 "What image processing techniques do you use?" Choice of commercial/open source/own software.

[[File:CCPiSlide4.JPG|500px]]

This was interesting and significant, as it showed that users were developing their own software for reconstruction and some filtering techniques but mainly using commercial software for segmentation and quantification.

Q5: "What are your current and future computational requirements? Please detail your current needs in terms e.g. of image size and number of images. What computer hardware is typically required to process your data and is fast turn around important to your experiments?" Comments given were:

[[File:CCPiSlide5.JPG|500px]]

As expected scale and size of object was important although these values can be represented within large (1/2 TB RAM) fat nodes so extreme cluster implementations were not necessarily there to be developed.

Q6 "Are there any algorithms in tomographic image reconstruction, analysis or quantification that you believe CCPi should consider providing an efficient open source implementation of?" Comments given were:

[[File:CCPiSlide6.JPG|500px]]

This gave a starting point for a list of potential new objectives for the SLA team to address.

Q7 "CCPi is involved in a range of activities to support the research community (see http://www.cpi.ac.uk). Of the current CCPi activities which do you believe to be the most beneficial to the research community? Please rate each area on a scale of 1 to 5, with 5 being most important."

[[File:CCPiSlide7.JPG|500px]]

The obvious priority was training requirements - and discussion in the WG involved ways to set up cross training facilities and access to data.

Q8 "Do you have any other comments on the CCPi project?" Comments given were:

[[File:CCPiSlide8.JPG|500px]]

== Results from Survey: July-December 2014 ==

Over the following six months changes have happened that have resulted in the following actions, (with illustrations):

* Quantification visualisation tools are now being developed for [[ImageJ]], [[Avizo]] and [[Paraview]] the most popular specific tools.

* Outreach events are continuing but at a lower effort level. This includes the Eurographics EuroVis conference

[[File:OutreachEvent1.jpg|500px]]

and different KTN Materials exhibitions that had significant attendance.

[[File:OutreachEvent2.jpg|500px]]

* Software show-and-tell events along with related seminars focus on guidance and use of the products as well as the research solution. These are now monthly events to keep a user and developer community connected. Last 18 events have posters with images; so for 2013 - 2014 season there have been the following events

[[File:Semseason2013_2014.jpg|600px]]

and for 2014 - 2015 seasons we have so far had,

[[File:Semseason2014_2015.jpg|300px]]

* Increase in cross training sessions: there is now a collaboration across Diamond Light Source / RCaH / University of Manchester and SCD/STFC labs with links to the Harwell Imaging Partnership and ISIS. Four annual training events are held at DLS/RCaH and four annual training events are held at UoM and specialist events are held at SCD/STFC in RAL all allowing for cross-attendance throughout the CCPi network. Last two specialist events were on 8 September 2014 with Ajay Limaya (CCPi Short-term fellowship) on Drishti 2.5 with a morning training session and a afternoon data analysis hands on session.

[[File:ExtraTraining1.jpg|500px]]

and 8 May 2014 an Avizo 'experts' workshop was held at RAL considering use of v8.1

[[File:ExtraTraining2.jpg|500px]]

* The survey allowed for representation at celebration events; including the Queen's Anniversary Prize at Manchester and the ToScA exhibition space where we sponsor poster prizes.

[[File:QueensAward.jpg|500px]]

ToScA annual meetings:

[[File:SpecialEvent1.jpg|500px]]

[[File:SpecialEvent2.jpg|500px]]

* Quarterly Developers' Workshop days held in CCP remote institutions on various specific topics; Birmingham, Nottingham and QMUL.

[[File:Dwdays.jpg|500px]]

ReportAndRecommendations

2015-01-29T14:46:17Z

Mcassdtm:

2015-01-29T12:21:57Z

Mcassdtm:

ReportAndRecommendations

2015-01-29T12:09:22Z

Mcassdtm:

2014-11-24T13:36:27Z

Mcassdtm:

2014-11-24T10:22:20Z

Mcassdtm:

===Comments on the respondents' second most used visualisation tool. ===

47 replies were given.

The most commonly used tools, with links to answers are:
#[[gnuplot]] 5 replies
#VMD 3 replies, but is not discussed in this section as the responses are extremely similar.
#Matlab 3 replies, but is not discussed in this section as the responses are extremely similar.
#[[Paraview]] 3 replies
#[[JMOL]] 2 replies
#[[VESTA]] 2 replies
#[[xmgrace]] 2 replies

Other tools had no more than a single response.

====Overall====
A few observations can be drawn from these responses:
#Users will prefer software that is written specifically for their domain of interest.
#Large datasets must be handled efficiently.
#Scripting or other ability to extend the tool is required.
#Publication quality output is a valued bonus.

Xmgrace

2014-11-21T17:01:56Z

Mcassdtm:

[[http://plasma-gate.weizmann.ac.il/Grace/ Xmgrace]] "is a WYSIWYG 2D plotting tool for the X Window System and M*tif. Grace runs on practically any version of Unix-like OS. As well, it has been successfully ported to VMS, OS/2, and Win9*/NT/2000/XP (some minor functionality may be missing, though)."

====Type of Data====

====Size of Data Set====

====Advantages====

====Quality of Documentation/Tutorials====

====Range of Data Formats====

====Level of Support Available====

====Performance====

====Limitations====

====Comments====