Recently, EDGE’s Fred Schwaller spoke with Joe Barnby, director of Senscapes. Senscapes is a music and arts project that creates immersive experiences of altered conscious states. These experiences take viewers through a perceptual and evocative journey of what it feels like to be in an altered conscious state, like during a psilocybin trip. Neuroscience research is at the heart of this work, where brain imaging data recorded from participants in research studies is converted into ambient music soundscapes, and then into visual representations of the music and the altered perceptual state. Here, Fred and Joe talk about the origins and future of Senscape projects.
Joe is a computational neuroscientist, currently doing his post-doctoral research in Australia at the Queensland Brain Institute. He is also a musician, and a founding member of Senscapes.
Fred is part of EDGE’s executive team. He is a science writer and an ex-neuroscientist based in Berlin, who also writes fiction and paints on the side.
Who and what is Senscapes?
Senscapes was founded by myself and Abbi Fletcher in London in 2016. Senscapes is all about creating immersive visual and musical experiences that put people in the centre of altered psychological states. We do this by taking brainwave data from human participants, and convert this data into music and art. More recently, we brought in Jerome Davenport to curate our operation in Australia.
In our first projects, we used brain imaging data from people participating in scientific studies who are in altered states of mind, for example after taking psilocybin (the active ingredient in magic mushrooms). We then converted these datasets into raw audio files, and ran them through synths and digital workstation software to make a track out of it. Then Abbie [Fletcher] would create a visual artistic representation from the music.
What you end up with is a visual and musical representation of brain activity in an altered state. Our shows were about bringing people into this experience, and we’d put them in the middle of a room full of projectors and speakers to immerse them in that altered state of mind the participants of the study felt when they’d taken psilocybin. Watch a discussion panel where Joe Barnby interviews scientists, clinicians, and participants that were part of the recent clinical trials using psilocybin to treat depression here.
Could you describe the process of turning brain imaging data into music?
Sure. This brain activity data is captured using Magnetoencephalography (MEG) during these altered states, and comes out as huge amounts of data code. We worked with scientists Steven Jerjian and Tim West to turn these 1’s and 0’s into sine waves.
Example of brain activity data measured using MEG.
We turn the MEG data into sounds by using scientific analysis software to export the analysed brain wave into a .WAV file. As both the MEG wave and an audio file are in sine waves, converting one to the other is only a matter of technicality. After we’ve exported the MEG signal as a .WAV file, I use a little creative license to turn it into a sound that can be heard using granular and additive synths.
You can listen to the 30 minute ambient mix title The Journey on Soundcloud.
What is about altered states of experience you are trying to communicate?
You can read about the core experiences of altered states of mind in literature and scientific articles, but this completely lacks the subjective experience of what it feels like. We want to recapitulate the feeling of altered states by using sound and vision. The aim is to capture you for enough time to take you out of your daily mind so you can experience aspects of a psilocybin trip, or something like a particularly abstract dream. The aim is to get at that emotional response, that feeling of ‘that’s all a bit strange’. We want to get underneath your boundaries to get you to a point which you might otherwise push away from but done in a way that (we think) is tasteful. We also wanted to communicate these altered states in a way where you can be awake and conscious, meaning you can take in abstract representations of what it is like to be in an altered state. And we wanted to make these experiences immersive, so that you’re not just looking at a piece of art, but moving past that subject-object relationship you often have in a gallery. I suppose you could say it’s what some forms of art try to get at, what good films and music try to get at this kind of abstract experiential way of looking at the world in a weird way.
Do you always base your pieces on altered states of perception, be they from dreams or psychedelic compounds, or are you also looking at altered states associated with mental health conditions, or even ‘unaltered’ states?
Senscapes is looking at how to evolve. The science-based content has become a bit more secondary than it was to start with. We’re trying two things: firstly, playing with the method of immersing people in an experience; and secondly, using beautiful experiences like natural environments to connect people emotionally with non-dream or non-drug-induced emotional states.
The first part is playing around with the immersive experience itself, and how we can tweak the experience of you being in an altered state regardless of what’s happening around you, so it can take you out of your thoughts and put you squarely in the moment. We’re playing around with the method of how to take people out of themselves, for example, by using subtle audio cues using 360 degrees of audio space, by placing visuals to cover the entire visual field, and play with set design to make the textures you sit or lie on weird and wonderful.
The second part is how to re-create authentically beautiful experiences. For that, we’ve been turning to immersive experiences of natural environments, using 360° cameras to film the Milky Way moving across the sky from dusk until dawn, and then creating a 30-minute piece with all the sounds and vividness of what it’s like to be in that experience. I currently live in Perth, Western Australian, and the outback is perfect for this. This is part of the same goal of Senscapes, which is to invite people to walk into a room that is geared to induce a strong feeling of being somewhere else, mentally and/or physically.
I hope Senscapes can branch out to focus on communicating the mind states of other people, not just ones that are somewhat familiar to us. I want to get to the point of us being the experts of sharing immersive experiences of other people’s states, not just our own.
You yourself are a neuroscientist. What do you research? How has your art helped your neuroscience research, and how has science helped your art?
I’m a cognitive/computational neuroscientist, and I’m interested in understanding the brain mechanisms which underlie how we communicate and learn from each other. I create computational models of behavioural data. I’m especially interested in how mental illness changes the way we might alter our decision making and beliefs. As an example, I’m researching how feeling unwell might change the way we form beliefs about who is and isn’t going to harm us. The bigger idea is that we can learn about our healthy interactions by studying when we aren’t so healthy.
Science and art have always been symbiotic to me. Ever since I was in my undergrad, I wanted to continue creating art because it brought something to me that science didn’t really tap into. Creating art gives me a more conceptual way of looking at things and gets me thinking about how to communicate concepts with often very minimalist forms. Having that part of myself helps me take my science more easily. Art also gives me a time to almost meditate with something that I can be in a moment with, and that allows me to do background thinking.
Where does art succeed in communicating concepts where science often fails?
I think art succeeds by giving wisdom to science. Science can be disconnected from those who aren’t experts. Art provides a framework to hang lots of scientific concepts on. From our exhibitions, I hope that people get absorbed into the feeling of what it’s like to experience something weird and abstract, and then go away and read more about the science behind it. I like to think the art is giving people a larger overview of what it’s like to be in different states, and then the science gives the details and the explanation. The same way that someone who has experienced a mental illness will understand the experience from a different perspective than those who study it as a science. Of course, you can be someone that studies the science of mental illness and also have experienced it first hand. The key thing is about working with those from different perspectives to ask the interesting questions.
