, New Urban Conditions

Neural Narratives: Collaborating with Machines, Ghosts, and Empty Architecture

Author: Tivon Rice

Merging the voice of a machine learning system trained on the works of J.G. Ballard with photogrammetric documentation of the gradual erosion of a modernist high-rise, artist Tivon Rice navigates the uncanny divide between human and machine cognition and explores the ghosts hiding in the cracks of urban spaces.


A kind of cinema-induced déjà vu

Even the run-down nature of the high-rise was a model of the world into which the future was carrying them, a landscape beyond technology where everything was either derelict or, more ambiguously, recombined in unexpected but more meaningful ways.”

High-Rise, J.G. Ballard (1975)

Beginning in late 2017, the demolition of the Netherland’s Central Bureau of Statistics office created an extreme type of slow cinema for railway passengers traveling in South Holland. Through the windowed frames of train cars moving past this scene, one could observe the building’s gradual erosion at the hand of a single excavator working from the top down. The following day, the image would be slightly, almost imperceptibly different as the concrete building slowly dissolved into the ground. Soon it will be replaced by another structure, which too, someday, will be erased.

I first witnessed this spectacle when traveling between my studio in Den Haag, north to Amsterdam. And it struck me with the sensation that I was encountering this scene for the first time, but it really wasn’t the first time. I was seeing it with my eyes, but also through the lenses of cinema, fiction, and all the other images that have been created around structures and scenarios like this – stories about architecture as psychological landscape and environments at the end of time.

I feel like I have seen this image before. What is this sensation, this kind of cinema-induced déjà vu?

“I think one’s sense of appearance is assaulted all the time by photography and by the film. So that, when one looks at something, one’s not only looking at it directly but one’s also looking at it through the assault that has already been made on one by photography and film.”

Interviews with Francis Bacon, David Sylvester (1966)

How have we been trained, through the closeup frame of cinema, to understand how emotions are registered on a human face? How have we learned about the body’s movement in space through the instantaneous, frozen moment of photography? And how do these images creep into our so-called “natural vision” during everyday encounters?

Still from Environment Built for Absence (an unofficial/artificial sequel to J.G. Ballard’s High-Rise) 2018

Today it seems our individual and collective sense of appearance is beset on all sides, as endless channels of digital, social, and televisual media join the cast of assailants that Bacon identified over 50 years ago. I understand an encounter with a new city or landscape because I have already visited that site on Google Maps. I know how and when to photograph a building because I have seen it tagged a million times on Instagram. I carry out these post-digital performative acts of making and consuming images, and with each reflection between natural and mediated appearance the feedback loop gets a bit more uncanny.


Prosthetic Vision

Returning to the Dutch statistics building, I imagined many ways of seeing this empty concrete shell. The train, reconfigured as a motorized camera on rails, provided one kind of image: a repeated panorama from right to left, channeling the penultimate scene from Tarkovsky’s Nostalghia. But perhaps a different kind of prosthetic vision would give me access to the complete picture.

Each month over the duration of the following year, I exited the train and walked to a hilltop across from the building (in Holland, a 15 meter-high mound is sufficient to be considered a hilltop). From there, an automated drone loaded with GPS coordinates followed a repeated pattern above the site, capturing 250 images each time. This specific photographic program—the flight path, the tilt of the camera, the exposure, the overlap of each image—effectively scanned the entire grounds, and over the course of 2018 created an archive of 3D models documenting the structure’s slow collapse.

Still from Environment Built for Absence

What did these photogrammetric images represent? From the train and from the drone, I observed this fenced-off section of the city from a distance. Back at my studio, however, I could remotely explore the building from the ground up—zoom in, look around corners, enhance. A virtual camera on the first floor provided closeup details of concrete rubble and piles of broken windows accumulating beneath the exterior. A wide-angle camera above the model framed the path of the excavator’s movements on the top floor. I could see the residue of the demolition crew’s labor, but the scenes were completely absent of any human form (photogrammetric scans generally won’t capture moving objects such as people, cars, or trees blowing in the wind). This absence and the hollowness of the gutted building were echoed by the logic of the 3D renders—glitchy surfaces with only an implication of presence or solid structures beneath.

As I continued to explore these empty virtual spaces, I found my internal narration once again assuming the voices of filmmakers and authors— artists who conjured similar images in their work, and thus impacted my perception of said images. The concrete, the building, and the desolate grounds all registered as settings for some unwritten Ballardian novel. How did J.G. Ballard craft such environments while exploring the psychological effects of urban decay, impending climate catastrophe, or social isolation in future landscapes? How would Ballard’s ghost navigate this terrain?

Not intending to directly answer these questions, but rather to investigate the mediatic voices that lurk around the periphery of my thoughts, I proceeded by training an Artificial Intelligence system to speak like J.G. Ballard.


Neural Storyteller

Given the complete corpus of his texts—over 1.5 million words from articles, novels, and short stories —this A.I., or recurrent neural network, emulates the vocabulary, style, and tone of Ballard’s writing. To further correlate this process with my images, the system takes a digital photo as input, uses computer vision to identify objects within the pixels, then describes what it sees using the language of the training data (Ballard). Originally developed by Jamie Kiros, Neural Storyteller was introduced to me in 2016 by her collaborator Yukun Zhu through the Artists + Machine Intelligence project at Google Arts & Culture.

Since beginning this dialogue with AMI, I’ve had many questions about human and machine perception, such as: When perceiving a situation, how do we draw upon our past experiences to understand that situation emotionally or rationally? In our post-digital lives how often are those experiences actually mediated experiences (if something we saw in a film or on the internet becomes our primary reference when encountering a situation out in the real world)? To what degree can a machine perceive a situation, drawing upon data rather than lived experiences? And can our observation of this machine perception allow us to reflect on human nature, perhaps even from a non-anthropocentric point of view? What can they tell us about us?

Sample output from Neural Storyteller

With these questions in mind, I showed the A.I. my photos of the half-demolished building to see if it could describe the materials, invisible bodies, and possible narratives residing within the broken concrete grounds. What stories would this machine tell me about the ghosts hiding in the cracks of urban spaces?


Environment Built for Absence (an unofficial/artificial sequel to J.G. Ballard’s “High-Rise”)

Installation view: Anamorphic Waves, London 2019

Excerpts from chapter 1:

A full view of a place, with a glass frame.

Signs on the street describe where things are.

Shops along the street with witty names and nice landscaping.

A mechanical bird in a cage.

The mechanical bird grasps to the cage and is reflected in a mirror.

And I am now surrounded by pale blue sky, with a silver volume suspended in the air.

It is a glass building near a freeway exit, a see-through building with a collection of machines inside.

How many times have I been here?

Installation view: Modern Body Festival, Den Haag 2018

In the middle of the structure, I gazed upward and reached into the mirror.

Once inside, it looked as if the entire world had been constructed a few hours ago.

There were many layers of glass and steel set up in endless rows.

And I ran my hand over each layer, as if to repair and protect them.

Catching sight of my appearance in the mirror, I gasped at the loss of my own features in the reflection.

Then I had a sudden recollection of what was going on here.

And with a snap of my fingers I was outside of the building as it began to fade into the late afternoon sunlight.

How many times have I been here?

Excerpts from chapter 2:

On the other side of the building, images of the city filled an otherwise empty courtyard: paintings of freeways and vacant parking lots, a long exposure photograph of an old street, large black canvases hanging in broken frames.

Even more signs were scattered on the concrete floor: a collection of briefcases, an intercom system, a pile of televisions, and an exact replica of this very building with smokestacks in the background and fake trees in the front.

Still from Environment Built for Absence

Taking a deep breath, I leaned forward and closed my eyes at the memory of those pictures, those signs.

The memory was an anchor, falling through the depth of the architecture, its chain cut by the surface of the building.

It fell deeper, deeper, deeper, and deeper, absorbing every object in every image.

And there was a long stretch of time before I finally reached the bottom.


Human/machine feedback loops

In pursuing a collaboration with the digital ghost of Ballard, I hoped to build a network of relationships between myself, the data (images and text), and these emerging artificial software agents. And as I watched the machine develop some kind of understanding about language, I found that my own understanding of language began to shift as well. I could see, from a sort of third-person perspective, how language evokes images and narrative decisions in my work. Sometimes, for A to better understand B, you need a C.

The feedback loop of image production/consumption that I mentioned earlier has become all the more uncanny as various manifestations of A.I. permeate our daily lives. They control how we receive information, they curate the images we see, and affect the images we create. This scenario seems very Ballardian indeed, and we can easily imagine a near-future when fewer territories in both the natural and built environments remain untouched by such digital systems of control.

If this is the case, what does it mean to work creatively with these machine learning systems?

Can a creative act also be a critical act, especially when liberating a technology like A.I. from its pervasive political or economic operations?

Rather than focusing on efficiency and performance, can we search for the poetic, the abstract, and the absurd nature of these systems? And in doing so, can we imagine and realize creative future relationships with the machines?

Environment Built for Absence

Environment Built for Absence was made possible by The Modern Body Festival, Yukun Zhu, Kenric McDowell, Google Artists + Machine Intelligence, Maxwell Forbes, and the University of Washington Center for Digital Arts and Experimental Media (DXARTS). Narration by Kevin Walton. More information at: tivonrice.com/absence

Tivon Rice

Artist and educator working at the intersection of visual studies and technology. Rice holds a PhD in Digital Art and Experimental Media from the University of Washington. He was a Fulbright scholar (Korea 2012) and one of the first individuals to collaborate with Google AMI—the Artists and Machine Intelligence research group.

If you noticed a typo or mistake, highlight it and send to us by pressing Ctrl+Enter.