What does research practice mean when it is unbound by the fusty, ring-fenced confines of academia? Reframing imaginative interpretations and the quest for animistic authenticity in a technologically moderated world, the designer and researcher Simone C Niquille and cultural producer Russel Hlongwane — both former alumni of the Nieuwe Instituut’s Research Fellowship — lay out their blurry methodologies.
FEDERICA ZAMBELETTI / KOOZ Let’s start with a broad question around this idea of the practice of research; how do you approach research, and what kinds of methodologies do you think are relevant today?
SIMONE C NIQUILLEWell — as always — it's a mash-up of things that I do; it's not as structured as I think the term “research methodology” implies, right? For me, it's much more like a visual observation, cross referencing something, and realising, hey, perhaps there's a bigger question here. It involves going with a gut feeling, and on that path, doing a bunch of things: one of them, for example, is interviews. I always really like to hold interviews and conversations. For example, if I read a research paper on computer vision and a training dataset that's been put together, I would reach out to the computer scientists who authored it, to ask them questions that go beyond the academic text. That, for me, is always inspirational — because you get the human side, the things that perhaps they've found frustrating or surprising. There’s the emotional side of the research question that I have as well; they tend to be about technologies, but I'm interested in the technology as a metaphor for human conditions.
Computer vision, in essence, is an elongated history of the way we represent each other; how we capture and reduce the world, to make it understandable. But still, it can feel quite abstract, so going to the people involved and having conversations is, I think, important. Then with the imagery, it might come from finding a strange video on YouTube where I'm not quite sure what the source is, but visually, it piques my attention. These things really happen in a non structured way; this sort of happenstance, digging into stuff, going into a tunnel and just trusting your gut; finding and pairing things with readings — things that we do anyways. It's really about cross referencing and visual connections; it's not something you can search for textually, nor can it be indexed; video is starting to be indexed, but not the visual part…
"Computer vision, in essence, is an elongated history of the way we represent each other; how we capture and reduce the world, to make it understandable."
- Simone C Niquille
RUSSEL HLONGWANEI oscillate between a mode of artistic research and a more speculative mode. In practical terms the speculative is at some point was a useful way — or maybe it depends on the question that one is asking — and one that seems specifically tied to the the perspective, the position and the context of being Black in South Africa, and the discontents with history or geography there. One gets the sense that the accounts of history — and the placements of Blackness in those accounts — are unreliable at both levels, from history with a big H that has been constructed outside of Blackness, flowing into Blackness, and the history that gets constructed within Blackness and flows outwards as well. I think one is trying to think about utilising that meta-narrative, using it in a very cynical or even intentionally callous way, to showcase and demonstrate its fickle nature. So in the projects that you would have seen, maybe there's something around ancient sites — the age-old sites of Mapungubwe — where these excavations have taken place and artefacts have been unearthed: there's so much mystery around this. How does one take the mystery to propose what else might have been excavated, what was not found, what still remains buried — and what remains buried in many other sites than we haven't encountered in this region?
It’s about using this point of consensus — like a site with this apparent mystery around it, yet which reveals tangible artifacts — to propose that if you can agree that something happened here, who's to say that other things didn't happen? Then we can constantly push that question further. In no way is anyone kind of providing a factual account, because history in itself cannot fully account for the facts that might have happened, you know? So this way of speculating upon history, speculating upon the present is always drawn out of some sort of consensus, whether it be social history or academic.
I also mentioned that the artistic research mode, which in my own practice, allows one to move beyond the confinement and the restrictions that academia often places on critical yet small gestures — such as citation and references. If one is investigating questions where the reference is not readily available and which has not been ‘authenticated’, where does one go? Then you have to refer to a conversation. But of course, that conversation does not carry the same weight as a piece of literature that has been cited a hundred times. So how does artistic research allow for a bit more flexibility, liberated beyond the mode of academic research?
"It’s about using this point of consensus — like a site with this apparent mystery around it, yet which reveals tangible artifacts — to propose that if you can agree that something happened here, who's to say that other things didn't happen?"
- Russel Hlongwane
SCNI love what you're saying about speculation; the tension it holds in contrast to ‘history’ is really important. One way artistic research can be interpreted is through this idea that it's about the future, like futurology. But I think speculation is so much more important in terms of things that seem set in stone as history or as fact; it doesn't really do it justice to constrain it to the things to come. It might also be about the things that happened — but it's about reimagining those. They will point to a different kind of future, once we've reimagined these pasts that we've taken for granted. But I find that incredibly important.
I was happy that you mentioned references; that sparked something for me. The work of the scholar Katherine Mckittrick and her writing — specifically the book Dear Sciences — gives so much weight to references and footnotes. She makes the point that there's a huge tradition, specifically in Black scholarship, Black art, Black culture, of not just being thankful but also acknowledging that there's a route to the things that you're making, that obviously they don't just sort of come out of thin air. On the level of graphic design, it might just be that your text is a certain size on an A4 or standard page, so that the footnotes are actually taking up more space, than that which we give our voices or the ‘main’ text, inserting reference as a way to make the main text seem plausible or perhaps important. I really love this idea of making and with that, being really aware that there are things that I'm building on top of, breaking this idea of a reference being such a specific thing, as it is in academia. This idea of needing to put the last name and the date — what if I don't have a last name? There are so many ways that you can easily break this convention of referencing. How are we going to think about these structures — ones that have been put in place for a certain reason, right? Referencing allows you to build on each other's research, but what if that doesn't fit into these sort of neat categories — apparently that then becomes artistic research? I would love for this work to also be counted as academic research, but as far as I've encountered, there are strict boundaries; for instance, something like a peer review journal has specific formalities around what you can publish. This is all to say that research often feels like a heavy word; like History with a capital H, so there is Research with a capital R. It doesn’t have to be, but I guess then we wouldn't have this conversation.
RHThanks for those points; I’m in complete agreement, that resonates absolutely.
KOOZ It's so interesting to consider the relevance of such methods today — of using speculative and artistic research as a means to question ‘fixed’ histories, to open up different perspectives and see where they might project us in the future. You both mentioned the importance of conversation or dialogue. Russell, the conversations that you host in your podcast often explore the potential of multidisciplinary exchange, can you expand on that?
RHI will try to use a practical example here. There’s a group of entities that are kind of found in the folklore of Zulu people, amongst others. It is through these entities that I grew a fascination and an interest in techno-politics, techno-poetics, science fiction and so on. So one figure — umuncwi — is this entity that is composed of spirit, matter and electric charge. My grandmother, for example, has encountered this figure, and there are a few others. These figures are crafts of transcendentalists from this region. Now, let’s suppose that there's a certain understanding of physics, of the elements and the laws of nature that these people, that this community of transcendentalists also understands in order for them to produce this entity — which, in my view, conjures the image of either the post human or the cyborg, right? It is through the figure of umuncwi that I enter science fiction. If there's a supposition that these people can understand very subtle and technical knowledge about physics, nature, then how might a theoretical physicist enter that conversation; how might they speak and respond to some of the propositions brought forth by this figure? How can we bring in a computer scientist to join that conversation, what can be translated through the gaps and the mistranslations?
Obviously, all of these people are speaking from different places of expertise, therefore they’re using a different set of vocabulary and language. I always find it interesting to pick up on those threads that cannot be translated all the way across — it's in those gaps that we're able to transcend the binaries and the power structures within various bodies of knowledge. What this allows for me to do, in terms of scholarship in South Africa, is to reject the idea that Western scholarship is superior to the indigenous one. At the same time, it's also to say that Western scholarship has taken humanity to a certain level; however, it's inaccurate and unreasonable to think that Western scholarship can account for the entire existence of not only humanity but also plants and animals, right? This transdisciplinary mode, working across various communities of practice, allows us to build much more wholesome knowledge projects — epistemic projects that work beyond the established binaries and hierarchies. That’s one value that I see. Then, if I take this process and go elsewhere, to someone who is interested in ancient civilisations, how can I take the mistranslations from South Africa, to see what tensions it might raise for someone thinking in that disciplinary approach? Through the attentive and sustained practice of these kinds of investigations, I believe this process will produce some interesting and novel, if not ‘credible’ knowledge, certainly some interesting thought experiments that can take the process of producing knowledge elsewhere.
"I always find it interesting to pick up on those threads that cannot be translated all the way across — it's in those gaps that we're able to transcend the binaries and the power structures within various bodies of knowledge."
- Russel Hlongwane
KOOZ Simone, conversations are also at the heart of your intuitive approach to research. Is that always a one-to-one exchange? Are there moments of exchanges with those that are not in focus, but rather who inform the research that you are undertaking?
SCNThere are different kinds of conversations. There are those where I'm curious to learn more — like the example I mentioned around research methodologies — where I reach out to someone to understand more than what is presented in a traditional format, like a research paper or a video, to learn about a process. Those tend to be one-on-one. In teaching, it's also important to use the classroom as a place for conversation, and that is an entirely different experience — not something that arises in terms of research methodologies, but these are important moments where you go through material with a group of students or a group of practitioners in different settings. For example, I have a research lab at the Design Academy in Eindhoven, as part of the Information Design master’s course, and there's more of a regularity to it. We meet every three weeks in workshops or group work sessions; there, it's about questioning material together, specifically software.
In terms of vocabulary, as Russell mentioned, I find that to be incredibly limiting, almost like a stopper for conversation. In the research conversations, where it's more one-on-one, I tend to be the one receiving information, but then in having a conversation with another person in another discipline, it's still me carrying that knowledge into questions.
I also realised that sometimes I have to use different words, even just in terms of how I describe myself, to get access? Like, if I'm going to write an email and describe myself as a designer slash researcher, sometimes the reply can be a no, because it doesn’t make sense to the person reading it. If I say I’m a visual cultures researcher, there's a play with words there that already happens, at the access to a conversation. It also happens within a conversation, where I realise that perhaps one person is using a word that is really technical for them, whereas in another context, it means something entirely different. There it's helpful to be a translator, almost — to come back to the Studio to make a glossary, in order to understand these processes and translate them.
KOOZ I love this idea of translations — it does resonate with the work that we do as researchers. And in this sense, you have both hinted at the kinds of research that you undertake, which evidently touches upon the intersections of identity and technology. How much and to what extent does your research agenda stems from your personal or professional experiences, and how have these developed as you have grown both as individuals and as practitioners?
RH In South Africa, a law was passed in the mid twentieth century, namely the Witchcraft Suppression Act. Essentially, this was the apartheid government demonising and criminalising the many rituals that were associated with and crucial to the world-making of Black folk. Through this act, there was a severance between society and a set of behaviours and rituals that have sustained for centuries. For me and my interest in technology, or technological thinking, all the politics entailed in this place is the most fertile, because it allows us to ponder and encounter the paranormal realm in ways that have been accessible [for centuries, which is] the rule of science fiction, in many ways.
I think this Witchcraft Act is tied to race, for example, in the many ways that Black studies have encountered and confronted the question of technology and race and the relationship between those things. From the perspective that I'm sitting at, this witchcraft is a place that one can dive into and try to excavate the material that has been lost. When one arrives at the rituals and practices that were banned or made illegal, how can one draw upon that and bring it into the world of science fiction or even “Afrofuturism”? How, then, does one use this future-facing discourse to respond to history?
SCN I'm going to start at another different end, to answer this question. For me, it's always been about the camera and photography; perhaps through education, it's become something else. It has taken me quite a while to come back to some of the curiosities I've had about the image.
"It was always very clear to me that an image is framed, that there's an elephant outside of the frame. Image is a construction, even if there is a factuality to it."
- Simone C Niquille
It was always very clear to me that an image is framed, that there's an elephant outside of the frame. Image is a construction, even if there is a factuality to it, but there is a clear ‘decision-tree’ that is involved with the one file, the one object with which you're being presented — and this is something that I was really incredibly drawn to from, from a really young age, actually. Going through art school, the one thing I would always detest was the idea of ‘personal work’. No, I don't want to make work about me. I want to make work and I don't want to be the subject. It took me a really long time to understand what the possibilities of that might be. Maybe to some extent, it’s difficult to entirely separate yourself from whatever you're going to make, but there's a huge spectrum in terms of what that means — from the self portrait to being part of a work in crumbs and bits and pieces. That spectrum has taken me a while to explore, and looking back is easier than looking forward, sometimes — at least on my own body of work.
But it's really clear how my own queerness has also driven an idea and an obsession with the image, right? The idea that you're being read by society; that certain attributes about you are going to define you in a certain way if you enter a room. Short hair, long hair, stupid stuff like that will allow people to read you by gender, by profession, by class, in very specific ways, whether it's conscious or unconscious. And that’s not even necessarily a bad thing. This is not about a binary, as in you shouldn't do this or that. It’s more to do with realising something that I am conscious of because of where I grew up, and the way I move through that place. It’s about realising that there are certain technologies that are specifically in this point in time, really enforcing this to a maximum. Really, the power structure that is societal and that has been in place. I'm specifically speaking about a central Europe — a very white place to grow up — which felt very incredibly coded, in terms of how you would present yourself. If you're going to dinner, you’d put on a different kind of dress, versus one you’d wear to school. Again, these are just visual attributes; we could go on about different kinds of ways that you can define or categorise a person. This goes south very quickly, and it is what we see today as well. But over time, this idea of personal work and queerness has converged, to the point where I finally understand why I'm doing a lot of this work. It’s against this idea of reading someone and reducing complexity, in saying yes, I can understand you, but understanding is about something else. It is not just about trying to pinpoint a person, but also a reality, a place, a history. I think this goes for many ways of being and of understanding a situation, at the time in which we find ourselves.
RHThanks for this. There's also something about how one tries to account for and render the ways that Black folk from this part of the globe have contributed to the fields of science, the fields of technology. However, one one would need to abandon the classical definition of science and technology, in order to engage with what the continent has produced and contributed to these fields. You would need to think through a much broader framework, and perhaps the way to think about technological practices from this location is through spirituality. For example, to work with the traditional healers, who understand the subtle qualities of plants and animals, how they work in the spiritual world. There's a whole other way in which Blackness requires the areas of technology and science to abandon the classical frameworks and asks to be read using a different set of lenses. This is maybe one way that I think about identity and the research questions that we take on, specifically around technology.
KOOZ If one starts looking at a little bit of your time at the Nieuwe Instituut, the research that you undertook involved two film works which draw on Zulu heritage, as a means of exploring the potential of Zulu imagination in relation to the fields of Architecture and technology. You just mentioned the idea of reframing knowledge systems; how did your research open up potential alternatives for your practice?
RHWell, it was the other way around. I was trying to advance one of those two films beyond what I had already produced. What the work at the Nieuwe Instituut produced was a really fertile and secure framework, a mode of thinking about how language serves as a repository to make a case. I was looking at a culture where history is not written — at least, the length of its history has not been documented in text form. It has been documented and carried through language. How does one work with the aesthetics and the composition of language — of terms, proverbs and phrases — to account for spatial imagination and technology? The question that I arrived with was, how have Zulu people contributed to spatial production; what has been the spatial imagination generated and valorised by these people? If one doesn't have history books to draw from, could I look at the terms that exist between rural areas and urban areas — how do these terms travel from one location to the other and vice versa? Which of these terms get carried over and which get left behind? Can I then use these terms and arrive at their etymology in order to understand something?
One term, for example, is ukunana. Ukunana is the act of asking a neighbour for a ration of food. If I don't have any potatoes, I can go to my neighbour and ask for potatoes. The term is devoid of any kind notion of debt and credit; it is to ask for food without shame. This term has evaporated in the urban context, yet it still remains in the rural areas. It tells us something about the social relations that have evolved, in response to and informed by space; how people have lived together, amongst one another. Another term is umsamo. Umsamo is an altar and is said to be the place that grounds a home in the spiritual realm. This is the place where one engages with the ancestors, for example. The question is, if the idea of umsamo originates in rural areas, what becomes a proxy for umsamo in urban areas? How does someone in a rural context elaborate upon umsamo, versus someone in the urban environments? What can we understand about the connection and triangulation between space, spirituality and the living entity?
"How does one work with the aesthetics and the composition of language — of terms, proverbs and phrases — to account for spatial imagination and technology?"
- Russel Hlongwane
I spent the time at Nieuwe Instituut trying to accumulate as many terms as possible, as a way to enter the world of spatial practices and technology. Out of this, there is a growing index — a vocabulary or glossary of terms that I'm building in order to think about these questions of architecture and spatial planning, as well as technology. I used the outcomes of that research and the methodology established there to get onto an MPhil or Master's program at the African Centre for Cities, which is on Southern urbanisms. I mention this because I don't have an undergraduate degree; I've never been to university or college, after high school. In South Africa, we have what is called ‘Recognition for Prior Learning’, which is to say they count your experience in the field as a proxy for an undergraduate degree. So this fellowship really allowed me to develop the kind of thinking that I could demonstrate and be accepted into the academy. That was a long way to answer your question!
KOOZ It’s fascinating. I'm curious to hear how your studies have enabled you to bring these research questions forward, and how things have evolved.
RHFirst and foremost, there is no concept of ownership amongst Zulu folk, in terms of a house or a home. The idea is that one never owns; one is a custodian. The colonial project — with the introduction of title deeds, as something that is tied to a market system of real estate — falls apart. This is still part of the tensions that we see playing out, in questions around urban planning and Blackness in South Africa; in ideas of statehood, regulation, and so on.
If I go back to the rural areas, I started to ask: if one is not an owner but a custodian, one is a custodian of what? We do learn that they say that you don't own the house; your house will be given to you by someone who comes after you. There's a way that one pays it backwards, if not forwards, right? Then they say, when one occupies a site, the first dwelling that you have to build is for your maternal grandmother. That’s followed by a house that is given to your paternal grandmother; then you build a house for your parents and so on and so forth. If you imagine rural areas — multiple huts and multiple home states — there's a sequence, there's a science behind how all of this comes together. At the front gate is often the kraal, which is where the cattle are kept — whether a house has cattle or not. This kraal has to be made, as this is a place where the ancestors rest. If you go further, they also talk about certain trees that one would plant to introduce as a part of making and building the house; the combination of animal life, plant life, spiritual life and human life, all of these things have to come together, to be held in place by this gravitational force that is the kraal.
One fascinating aspect I also got to understand is that the orientation of the household is equally important. Homesteads face eastwards, as a place of life, of life-giving. When you are placed to rest in death, the head faces westwards. So there's this orientation that is often assumed to be primitive, but which has not been attended to accurately with the kind of sensitivity that it requires. This is what my masters’ studies are slowly trying to unravel and pick out.
"There is no concept of ownership amongst Zulu folk, in terms of a house or a home. The idea is that one never owns; one is a custodian. The colonial project falls apart."
- Russel Hlongwane
KOOZ At the Nieuwe Instituut, you were looking at the presence of certain words; either they were shifting from the rural context into the urban or vice versa. I was wondering if those ideas, words or their absence continue to inform what you’re looking into now?
RHI think I've moved from words to rituals. Or maybe not — I've accumulated a method of working with words, and now I'm applying the same framework to think about rituals, to think about animal life and plant life, also. It's really taking a small slice of what I developed at the Nieuwe Instituut, and expanding it into other aspects of Black life in South Africa, you know. Part of my studies also involves thinking through the migration of Black folk from rural to urban areas as a forced migration. There was not the time to think about how we might translate and take elements that held life in rural areas into urban areas. In my generation, we are thinking about how and what to negotiate and carry over whilst holding the integrity of these practices and these terms, their social relations.
KOOZ It's really fascinating. Simone, you're exploring a very different kind of space in terms of the one that bodies inhabit in relation to technology. That's something that you started working on well before your fellowship at the Nieuwe Instituut; how did your fellowship inform your explorations — especially doing so during the 2016 elections, when Hillary Clinton ran for the presidency.
SCN It's difficult not to be inspired by Russell's answer, because it really ties into the work I've been doing, particularly in the last four years, so I'll try to get back to that. To answer your question about the Nieuwe Instituut, this was almost ten years ago, I’ll try to condense my answer. I had been spending time on my master’s studies between 2011–2013. I graduated with a critical work on facial recognition as convenient yet privacy-corroding technology; I'm mentioning this because it was such a particular and important time where on the one hand I had to explain what facial recognition was, constantly — through, for example, a new feature of Facebook, which offered to tag your friends and link to their profile. At the same time, during that spring and summer, the Edward Snowden papers were leaked, which transformed consciousness of what it means to contribute to social media platforms or technology at large; the idea of a surveillance complex really entered a public consciousness — like, hey, this is what we're building and contributing to on a large scale, without even knowing it. That realisation also really changed this idea of facial recognition from something that might be convenient to something that has this very dark under-layer, which might actually have spawned its creation, right?
Going back in history, in terms of why you would even want to identify someone, you quickly arrive at pseudoscience. For example, a person called Johann Kaspar Lavater, who was a 18th century Swiss priest who was very interested in figuring out who you were by your looks, like physiognomy — the idea that you can read a face and deduce information like intelligence or criminality. We can clearly see how some of these ideas have informed face recognition to this day. In 2013, it was really important for me to talk about this tension between surveillance and convenience; at what point do you subscribe to the same technology that at some point in time, you would consider to be absolutely the worst, most intrusive thing ever. Ten years later — and to my own shock — I'm using face recognition to unlock my phone, whereas if you had asked me in 2013 or even 2016, I would have said this would never happen.
"It was really important for me to talk about this tension between surveillance and convenience; at what point do you subscribe to the same technology that at some point in time, you would consider to be absolutely the worst, most intrusive thing ever."
- Simone C Niquille
So there's a human side to this, a slow seeping of these technologies that enter daily life, and your own threshold changes with it, to a certain extent. It might just be practical: I don't like using WhatsApp, but a lot of my kids’ friends’ parents use it. So if you’re trying to make a playdate, you need to use that platform. There are all sorts of complexities and reasons to negotiate with these identifying technologies. My time at the Nieuwe Instituut in 2016 was a follow up from that research on face recognition; I was deeply interested in the profession of lookalikes, people who earn money because you look like someone else. Specifically in the United States, this might be someone that looks like a popstar, that you could book for your birthday to do a concert or just make an appearance. It really sort of functions through the principle of simulation, literally banking or profiting off this idea of representation. There is a field of celebrity lookalikes, but also this strange grey area where perhaps a politician is also a celebrity. This was the case of Hillary Clinton, who came to be known to the public as the First Lady, as the partner of former President Bill Clinton when he took office in 1993. Of course, her own career saw her taking political positions over time, resulting ultimately in her own bid for the US presidency in 2016 against Donald Trump — which was also a bizarre pairing. Among other things, I used my time at the Nieuwe Instituut to make a documentary film following Teresa Barnwell, a Hillary Clinton lookalike, in the run up to the US elections — not to speak about face recognition in a technical way, but rather from a human perspective. To understand what it means to be read, to be recognised as a person that you’re not, and what the consequences are on your life and your being. This was so much more complex. Teresa has enjoyed her career very much, but in October of 2016 it wasn't an enjoyment at all. It was a huge burden, where she would get pulled into tabloid media stories; it said that she was hired by the government, that she works for the Pentagon, and that she was an official lookalike of Hillary Clinton — there was a media frenzy, trying to unpick what might be real or not. The idea of fake news was really happening then, things were unraveling in real time. There were crazy stories and photo shoots with the real Julian Assange, but as the fake Hillary Clinton. It was a weird meeting of reconstructed image reality; what does it mean to create a factual image or to create something that's newsworthy, when that’s not the same thing.
In the film, you see her speaking about her life and work, essentially. There's also a moment where I ask her if she wanted to be 3D -scanned. I had her 3D scanned by a place in Los Angeles, where she lived, and I would also use the 3D avatar in the video. Part of that process was the question about what rights you might have to your own image? Whereas in photography, there is such a thing as the right to your own image, in the world of 3D assets that doesn't exist. If I'm going to scan myself, it's mine, but in the case of Teresa/Hillary, I have a contract with the 3D scanning company and the file is mine, because I’m the one who commissioned the scan — which is bizarre. I mean, this file should be hers. Moreover, a scan is so much more potent than an image, right? I can manipulate this scan to a larger extent than I could a two dimensional photograph. This was one of the questions that was part of the film, which then carried into other works. This conversation could go on for a while, since we're talking about a 10 year span, and all the stuff that spawned out of that time, but the film The Fragility of Life and the work with Teresa really encapsulates my time as a fellow..
KOOZ Do you think it has seeped into your current research on synthetic training data for domestic computer vision? First of all, I’d love you to clarify exactly what that means, and then to understand the research that you're doing today within the context of university and pedagogy, of working with students.
SCN I think there was a bit of a bridging project, which was called Safety Measures; it was shown at the Dutch pavilion at the Venice Biennale of Architecture in 2018. For me, it was partly a shock to be part of an architecture biennial — and partly amazing, because it allowed this work (which mostly resides in video) to develop into spatial forms. It was a work on ergonomic design software, which came out of the research I was doing on 3D avatars. So this question of scanning a Hillary Clinton lookalike and the ownership of this data transformed into research on how digital bodies come to be. What was its technological history; who was the first person who felt the need to create a digital representation of a human body on the computer screen. Why and how did that come about? That was actually in 1989, at University of Pennsylvania; a professor called Norman Badler — at least, this is the event that's recorded, speaking to Russell's point. I'm sure that a lot of this technology emerged in many places simultaneously, but the first ‘avatar’ or digital human that I could find is called Jack. Norman Badler was interested in creating a digital figure for animation; digital puppetry is what it was called then, but he needed some data to inform the literal 3D coordinates, and this would involve inserting coordinates to create meshes or geometric forms on the screen. One resource that he found to be incredibly useful was a body measurement data set by the US Army that was collected in the 1980s — the army was interested in this because they had to create standardised uniforms and gas masks for a large cut of the population. This sort of thing is always referenced for things like the design of cockpits for fighter jets; all these places that were spatial, that were either housing the body or enveloping it in some way, whether it's clothing or machinery, where you need to know the volume of the human body in an average sense. They measured soldiers and created this sort of average body; Badler then used that to create this digital avatar called Jack.
Jack has now become an ergonomic design software; it's currently owned by Siemens, you can still buy it and it’s still called Jack. One thing I was deeply interested in is what the software looks like? At some point, there is an interface where you have a drop-down menu and you can make your own Jack. But what are the data sets that inform this body? Of course, it's incredibly racist, as well as incredibly misleading in the way that the interface is representing the data behind it. For example, one category would be Chinese, but what does that mean? Are we talking about a nationality or a geographic area? It’s incredibly reductive of a complexity that is never addressed. I was really interested in these data sets and how they're produced, in order to create these representations on screen that seem somehow reassuring? As a designer, I would place my design for an automotive or a factory floor into the software; based on a profile, I should see if a certain kind of person could then reach a button. But it was very unclear — to me, at least — how this profile is made. Yes, it's technically an average US citizen, but what does that mean; who was measured to create the average US citizen?
"There are many different manifestations of home, of being, of living in the world, but what technology is doing is reducing it to one or another certain kind of being. I don't believe that I am required, as a designer, to produce a reaction to that."
- Simone C Niquille
Through a bunch of loopholes, this led me to work I do now, which follows an interest in training data to create models of understanding and to test the limits of their knowledge. What this means for computer vision and domestic space — which is always such a mouthful — is essentially as follows. This kind of vision is already in use today, for instance the sensors and cameras that will enter a room in a vacuum cleaner that moves around your house. This vacuum cleaner needs to identify the objects that the camera encounters; it needs to ‘understand’ the environment that it's in and make a decision on how to behave within that environment. Domestic space is incredibly interesting, because there is not enough visual data online to usefully scrape. For facial recognition, you need images of faces — so Facebook, of course, was fantastic; as most people post photos of themselves, that auto generates a training data set. Whereas for a home, where do you go to gather that data? Synthetic data became one way to answer that lack, where we sort of looked at industries like architecture and product design, which use CAD or Rhino-based software in their design process. Couldn't that three dimensional virtual data be used to assemble spaces that then we call homes? If we photograph those virtually, then those digital photographs or renders could become the image-training data sets.
In a really long, meandering way — but to hook on to what Russell was saying — as a designer, I'm often asked why I am making work about it rather than countering it, right? Like, why are you saying that the way that we're teaching computer vision what a home is, is so hyper standardised? Why is a chair such a very specific kind of chair with four legs and a back, rather than an object for sitting, which could be many different things — it could be a log, or it could be a carpet. The interesting thing is that the critique, apparently, or asking the question isn't always enough. Sometimes it seems as if you also have to produce an answer. Personally, I feel that no, I don't have to produce the answer. The answer is there. The point is that there are many different manifestations of home, of being, of living in the world, but what technology is doing is reducing it to one or another certain kind of being. I don't believe that I am required, as a designer, to produce a reaction to that. I really see my role as the person digging through these technologies as a designer, using the software and technologies that I'm interrogating and through those processes and workflow, to hook onto something else. For example, this interface in the ergonomic software, where I realised that using the drop-down menu is weird. That's the sort of question that drives my research and design. It isn't so much creating these other worlds — because I truly believe those exist, but I also don't think that we need to go and capture them; that's not the answer. We don't need to make a training data set that encapsulates the entire world. It's more about wondering why we are even making this technology and for whom — by whom? That's often when it falls apart, because it's not supposed to be for the community. It's for profit. It's for a general market; it's for sale.
RHIt’s so rich. Wow, wow.
KOOZ Honestly, looking at the kind of evolution of your research and how it has really kind of taken you in different directions, but somehow always sticking to a certain kind of core of inquiry — it’s really impressive. And it's been a treat to listen to you both. Indeed it has been really rich in terms of understanding research can help us grapple with our own questions, and through which we can really find an understanding of where we are, where we come from, and how we can shape different narratives around our own identities.
RHThank you to Simone; it was really beautiful to share space with you and get into your brain so thanks for sharing so generously.
SCNYes, absolutely. I'm so glad we made this happen; thank you so much Russell and also Federica, for guiding us through the conversation.
KOOZ Thank you both, truly.
Bios
Simone C Niquille is a Swiss designer and researcher based in Amsterdam NL. Through technoflesh Studio she produces films and writing that investigate computation as the new optics. Her work is concerned with vision technologies, the images they make and the worlds they create - from computer vision, 3d animation, computational photography to synthetic training datasets. Her work advocates for non-binary technology and against machine learning as a tool to validate and instrumentalise assumptions and reduce reality.
Russel Hlongwane is a cultural producer based in Durban, South Africa. His work is located at the intersection of heritage/modernity and culture/tradition as they apply to Black life. His said practice includes artistic research, film, creative producing, design theory, curatorship, writing and performance – often against the backdrop of installation form. As a consultant, he works with cultural institutions and government departments concerned with providing meaningful support to creative ecologies. He has served on boards of many organisations in the sector, currently on the Prince Claus International Advisory Committee (2023-2025). He is currently (2024) pursuing a MPhil in Southern Urbanism at the African Centre for Cities at the University of Cape Town.
Federica Zambeletti is the founder and creative director of KoozArch. She is an architect, researcher and storyteller whose interests lie at the intersection of art, architecture and regenerative practices. Prior to dedicating her full attention to KoozArch in 2024, Federica collaborated with the architecture studio and non-profit agency for change UNA/UNLESS working on numerous cultural projects and the research of "Antarctic Resolution".