by Ashley Hay
In certain circumstances, something that has been seen or experienced may not be accessible to the conscious mind of the person who saw or experienced it. The brain has means at its disposal to protect the psyche. Received visual stimulus too traumatic to contemplate can be erased or altered in the memory. An unthinkable image may be substituted for something else.
I saw a brilliant white flash bounce against my windscreen. In 1886, Jules Verne was shot twice by his nephew, Gaston.
To be precise, Gaston shot him twice in succession, as opposed to shooting him on two separate occasions. Jules Verne was not fatally wounded. Though left with a limp, he otherwise made a full recovery. He went on to write a further 25 novels. In 1888, he stood for and was elected town councillor for Amiens and served in that position for 15 years. He died at home in 1905 of diabetes-related illness with his wife and son by his side. The house is now the Jules Verne Museum. It is not a golf club.
The eye in the sand
Pitch fever
The CAVE artists
Dyani Lewis
In the era of big data, biomedical databases are brimming with protein structures, image collections and genomic sequences. As the data mount, new ‘cave automatic virtual environments’, or CAVEs, are being built to help researchers pick through the files. Walking into the CAVE2 at Monash University in Melbourne, the first thing that strikes you is its immensity. What appears to be an enormous electronic billboard encircles the space, forming a cylindrical room with a 24-foot diameter. The images displayed on the 80 high-definition liquid-crystal display panels beam out at a crisp 84-megapixel resolution. And with a pair of stereo glasses, they pop out of the eight-foot-high display wall in three dimensions.
In the virtual driver’s seat is David Barnes, a radio astronomer by training who runs this new CAVE2 facility. He holds what looks like a chunky television remote controller with four lollipop-like baubles attached to the top. This wand, together with the similarly adorned ‘reindeer glasses’ and motiontracking sensors, control navigation so that Barnes can fly across the surface of Mars, explore an Egyptian temple or examine pathways of neural activity.
CAVE2 is state of the art in electronic engineering and computer visualisation technologies – and Monash’s CAVE2, which opened its doors in late 2013, is one of just two such facilities in the world. The first, slightly smaller CAVE2, with 72 panels instead of 80, started accepting scientists at the University of Illinois at Chicago (UIC) in October 2012. The circular CAVE2 design of both facilities is the latest immersive virtual environment to emerge from UIC’s Electronic Visualization Laboratory (EVL).
Proponents of these ‘cave automatic virtual environments’ say they could be a boon for basic research scientists wanting to visualise complex 3D structures – from elaborate molecules to whole organs to networks of gene or protein interactions. CAVE2 ‘will push some science forward,’ Barnes says. ‘It’s a matter of finding out the right way to apply it.’
* * * * *
Olusola Ajilore is finding a way. A neuropsychiatrist at UIC, Ajilore is using the CAVE2 in Chicago to study whether impairments in white matter integrity underlie depression in the elderly. Within the immersive 3D lab, he can virtually step into and walk through the brains of his research participants, or at least their diffusion tensor images. ‘It looks really cool,’ says Ajilore – so cool, in fact, that across the world Barnes is using Ajilore’s brain ‘connectome’ images to show off the potential of the newly minted CAVE2 in Melbourne.
Standing inside the connectome is like being in the middle of a large tangle of multicoloured electrical cables. Long fingers of green project out from the display, blue cables fall like curtains to the side and a thick tangle of red hovers near the centre. Zoom out and meaning starts to emerge from the chaos. The red is soon recognisable as the corpus callosum, the fleshy junction relaying information laterally between the brain’s two hemispheres, and the green and blue cabling indicate front-back and top-bottom flows of neural activity. In an instant, the computer-generated colors have decoded and simplified the complicated highways of the mind.
In November 2013, Ajilore presented his connectome work in a poster at the Society for Neuroscience conference in San Diego. Reactions ranged from ‘Is this art?’ to ‘This should be a TED talk.’ But more important than its visual magnificence is what Ajilore hopes to detect in the connectome when he’s in the CAVE2. ‘We might be able to appreciate more subtle differences that aren’t detectable when you’re looking at what is essentially three-dimensional data in two dimensions,’ he says.
The predecessor of the CAVE2s in Melbourne and Chicago was UIC’s first CAVE, a square room, 10 feet across, with graphics projected onto three walls and the floor. Built in 1991, the original CAVE was named for its obvious grotto-like qualities, but it also nodded to Plato’s cave, the allegorical place where shadows become reality for those inside.
Since its inception, variations on this basic CAVE design have cropped up all over the place. There are CAVEs now in museums, architectural firms, car manufacturing companies, government laboratories and universities. Developers have also tinkered with the construction, creating cubic models that completely surround the user, as well as spherical and pentagonal enclosures. And as display technologies have improved, projector-based CAVEs have been joined by systems such as Nex-CAVE and CAVE2, made of high-definition, 3D-enabled panels.
But as commonplace as CAVEs have become, the use of these multimillion-dollar visualisation facilities as research tools has so far left scant trace in the biomedical record. Most reported applications of virtual reality environments in the life sciences literature focus on their use in medical training or as a virtual environment to encourage patient rehabilitation – not drug development, even though CAVEs are often touted for this research application.
As such, the question of whether scientists and their institutions have managed to turn infrastructure spending on CAVEs into biomedical advances – let alone whether newer CAVE designs are worth the investment today – is an open one. Monash’s CAVE2, for example, had a price tag of $2 million, not taking into account construction costs to accommodate it or the operational costs to fund software development and personnel, which are often recouped through hiring fees in the hundreds of dollars per session.
* * * * *
Harel Weinstein is one scientist who sees value for money in CAVEs. In 2007, he and his colleagues at the Weill Cornell Medical College in New York were trying to nail down how the addictive drug cocaine interacts with its target, a protein that normally pumps the neurotransmitter dopamine out of the synaptic clefts between neurons for reuse. ‘We went into the CAVE’ – the original square-room kind – ‘and we positioned the cocaine where our computation said it would sit,’ Weinstein recalls. ‘And then we said, “What can we do to prove that it sits there and nowhere else?”’
When viewing a simulation of the subtle, yet dynamic, flexing and bending of the dopamine transporter, Weinstein and his team noticed that just above where cocaine bound sat two helical structures that waggled into close proximity to each other – a feature that had not been obvious from looking at the interactions on a two-dimensional desktop computer. Knowing this, the researchers devised a way to reversibly cross-link these two helices in an engineered version of the dopamine transporter. Using this molecular clamp to lock in or lock out cocaine molecules in tissue culture cells expressing the engineered protein, they showed that the cocaine bound to the transporter at precisely the place where the computer modelling had predicted.
Without identifying the helices in the CAVE, ‘this idea would never have come to us,’ says Weinstein. Importantly, the work also confirmed a suspicion that Weinstein had had all along: that cocaine uses the same binding pocket on the transporter as dopamine. Efforts to find a drug therapy for cocaine addiction would now have to steer clear of simply looking for an inhibitor of cocaine binding, lest it also disrupt vital dopamine function. ‘That was a revelation.’
>
Robbert Creton, a developmental biologist at Brown University in Providence, Rhode Island, had a similar ‘aha’ moment when he stepped into his institution’s CAVE to look at confocal microscopy images of a developing zebrafish embryo. Creton was trying to determine whether a sac-like organ, known as a Kupffer’s vesicle, which develops around 12 hours after fertilisation, could be responsible for establishing left–right asymmetry in the growing fish embryo.
Creton had hypothesised, but hadn’t been able to prove, that the mechanism could be related to the distribution of tiny hair-like cilia that project into the fluid-filled sac. On his computer monitor, the cilia on one surface of the vesicle had been hard to distinguish from those on the opposite surface, and quantifying differences in density between the two surfaces had been impossible.
‘The moment that we stepped in the CAVE, it was pretty obvious,’ says Creton. What he saw there, but had failed to see on a flat screen, was that the cilia were indeed unevenly distributed throughout the vesicle. This explained how they could control the flow of fluid to establish a left–right chemical gradient that affected zebrafish development.
In his current work, Weinstein is interrogating more complex molecular interactions, involving ‘a whole environment of machinery’, including multiple proteins as well as the biological membranes they associate with. These systems, he says, are ‘even more difficult to fathom’ without an appropriate visualisation facility. By building 3D models of these complex molecular networks, Weinstein is able to ask questions and use ‘what if ’ scenarios to direct wet-lab investigations, as he did for the dopamine transporter. ‘We say, “It looks like this thing is involved, so what if we mutate it, or what if we change it, or what if the membrane now is more rigid than it was before?”’
It’s in these increasingly complex systems that advocates of CAVEs see their greatest value. Jürgen Schulze, a computer scientist who writes visualisation software for CAVEs at the California Institute for Telecommunications and Information Technology (Calit2) in San Diego, says that when it comes to sifting through large data sets, computer algorithms and supercomputing facilities are often enlisted to do much of the heavy lifting. But large-format 3D displays put the onus back on the researchers’ own visual acuity to explore the data and extract meaning.
‘You can’t just tell a computer, “Find patterns,”’ Schulze notes. ‘You have to tell it what kind of patterns you’re looking for, and then you’ll only find those patterns.’ With the CAVE, he says, ‘our eyes just see patterns. You don’t have to make any effort to do that.’
* * * * *
When the first CAVEs began appearing on university campuses, large-scale visualisation was often the endpoint of research studies. Scientists would demonstrate structures to students and colleagues but rarely actively investigate those structures. Nowadays, visualisation is designed to be part of the research process – with each viewing prompting new questions and ways to sort and combine data sets. ‘You get in this very rapid iteration process,’ says Larry Smarr, director of Calit2. ‘It’s like climbing a tree. You don’t just jump to the top of the tree. You go branch by branch. And you have to, at each branch, figure out where to go to next.’ With their spacious interiors that can accommodate more than a dozen viewers, the CAVEs also generate a collaborative experience, he says.
Not everyone, however, is convinced that CAVEs are changing research so dramatically. Philip Bourne is a computational biologist who used a CAVE ‘on and off ’ at his former institution, the University of California, San Diego. He found the immersive environment to be particularly useful when dealing with ‘very large and very complex’ molecules, such as ribosomes. But he now sees CAVEs as being of most benefit in education, not research. ‘I couldn’t point to a publication of ours and say, “This piece of science was done because of the CAVE,”’ says Bourne, who in March 2014 became the first associate director for data science at the US National Institutes of Health in Bethesda, Maryland.
Drew Berry, a biomedical animator at the Walter and Eliza Hall Institute of Medical Research in Melbourne, is another who’s sceptical of CAVEs’ value for research. For those who do need to really get inside their data, Berry’s money is on head-mounted devices. Once virtual reality goggles come down in price, ‘you can essentially create a virtual CAVE for $300,’ he says.
Already today, there’s a small desktop-sized visualisation unit called FluidVis that’s commercially available. A combination of a 70-inch 3D display, special glasses and proprietary software, FluidVis, from Fluidity Software of Somerville, Massachusetts, retails for around US$25 000 – a fraction of the cost of a fullsized CAVE – while still capturing ‘a lot of the immersive benefits’, according to Andrew Forsberg, a Fluidity cofounder and research scientist at Brown University’s Visualization Research Lab. FluidVis also has an added benefit of accessibility, notes Creton, who has used the tool. ‘It’s more convenient because it’s right in the lab,’ he says.
Competing – and cheaper – systems could be built with software developed by Zeynep Gümüs¸, a computational biologist at Weill Cornell. Gümüs¸ had used her institution’s CAVE to identify genes specifically involved in mouth cancer development and to find regions of noncoding DNA that help drive cancer more generally. After being approached at conferences by scientists who told her they would conduct similar network visualisations but for the lack of a CAVE, Gümüs¸ and her graduate student Vaja Liluashvili designed iCAVE – the interactome CAVE. According to Gümüs¸, this software works just as well in a CAVE as on a desktop system assembled from off-the-shelf components that generally cost less than US$2000.
But price and infrastructure hurdles aside, perhaps the biggest challenge for CAVE2s and other full-size visualisation facilities is awareness – ‘people not knowing what is possible,’ as Andy Johnson, head of research at EVL, puts it. Johnson doesn’t see this problem as insurmountable, though. ‘Once people see an example that’s close enough to what they want to do, it starts to click and they start getting ideas,’ he says.
Scientists like Ajilore are only too happy to be leading the charge. ‘I think there’s a huge upside in having new ways of visualising large data sets,’ he says. ‘If we have innovative ways of being able to visualise that data and understand that data, hopefully that will lead to better discoveries.’
Life, the universe and Boolardy
The quantum spinmeister
High-tech treasure hunt
Sarah Kellett
Geologist Steve Hill moves through the outback, collecting leaves and looking for a sign. ‘What you see of a tree on the surface of the land is only the tip of the iceberg,’ says Hill, director of the Geological Survey in South Australia. ‘Tree roots extend down very deep to bring up water, particularly in arid parts of Australia. When they bring the water up, they’re also sucking up a lot of the chemicals that are coming out of the rocks. We then sample the leaves and send them off to a chemical laboratory for analysis.’
Eucalypt roots can extend 30 metres down into the ground – the height of a 10-storey building. Through the process of transpiration, water is drawn up from the roots and transported in the xylem to small pores in the leaves, where it evaporates.
As well as gold, uranium, lead, zinc and copper, geologists are looking closely at leaves for elements that indicate mineral deposits might be nearby, such as buried granite, basalt and quartz vein host rocks. Mining companies are interested in this technique as a cheap alternative to drilling a hole for exploration, which costs thousands of dollars. Analysing a plant sample only costs around $40 – and it’s much gentler on the ecosystem.
River red gums and mulga trees are popular species for sampling, as is grassy spinifex.
‘Spinifex has been incredible for us,’ says Hill. ‘Although it’s a little spiky grass at the land surface, [it lives] to be hundreds of years old and sends roots down very, very deep.’
The quantity of gold in a single leaf is tiny. So, at first, scientis
ts couldn’t be absolutely sure whether it was being drawn up from below or had just blown in on the wind. Mel Lintern at the CSIRO set out with a team to find conclusive evidence of the origin of leaf gold.
‘Of course we can’t see the gold in the leaves with the naked eye,’ says Lintern. ‘This is where we needed the analytical powers of the Maia mapper at the Australian Synchrotron. This remarkable machine – the size of a small cricket oval – was able to show us that the gold was actually within the plant rather than stuck on the outside as dust.’
The 3D images revealed tiny nuggets within the leaves. ‘We all know gold produces nuggets, but the nugget effect actually carries on into the vegetation,’ says Lintern.
Gold isn’t distributed evenly throughout the tree, but instead is dotted around; so, some leaves have more than others. To best detect gold, the research suggests geologists will need to sample all over the tree to get a nice average.
The team also found that gold seeps out onto to the surface of the leaf, and may be washed away by rain. Steve Hill has also noticed that they detect less gold in leaves after rainfall.
* * * * *
Animals can make handy prospectors too. The compass termite cleverly angles its nests to stay cool during summer and warm in winter. The cooling and ventilating tricks of termite mounds have inspired architects for years. Now they are helping geologists, because their masterpieces can be decorated with traces of gold.
‘The termites burrow down to the water table and also eat spinifex grass and cycle those chemical elements into the termite mound,’ says Steve Hill. ‘We can take a little chip off the termite mound and analyse that, and that can tell us lots about what’s happening underneath.’
The termites lose a bit of their house in the process, but can rebuild it within a few hours. And compared to a drilling operation, the impact is tiny.
