chapter

WP3:_SMALL_FILE_PUBLISHING_Internal_pad

**Notes Pad - WP3_Internal Work Session 1 **

This is the pad for our first internal work session. This session will be split into two parts:

Part 1 with Radek and Laura: Tuesday May 26 from 16-19h (Amsterdam time)

Part 2 with Joni and Laura: **Tuesday June 16 from 16-19h (Amsterdam time) **

The sessions will take place online in Jitsi here: https://meet.greenhost.net/connecting-otherwise

Part 1 -

Presentation and workshop from Radek and Laura of the environmental impact of streaming and the research SFMF have done around this topic, as well as a more hands-on workshop on different methods and options for compression, and the choices that are made (i.e. learning about data moshing techniques and open source programs that can be used for making small files). Following this part we then have time to try out the different methods Radek showed, on our own time, and document them, so we can discuss them together in Part 2.

Notes and documentation from workshop / presentation

**Participants: **

Laura

Mark

JK

Juliette

Radek

Margarita

Introduction of Jan Kees

• Composer by training, Tv broadcast via internet

• Lot of radio, Icecast, Liquidsoap

• Interested in broadcast and livestreaming regarding small file

• reduction of latency, gap between video meeting immediacy and live broadcast

• WebRTC and even faster stuff by the guz who developed VLC

• very low frame rate video broadcast, audio is immediate thats enough to perceive well

• PUB rietveld

**Laura presentation / introduction on the different actors involved in the carbon emissions of streaming **

• Laura shares screen, checking in from Texas

• best practices, decreasing carbon emissions of information and communication technologies

• Laura is calculating that global greenhouse emissions from streaming are 5.7% (this includes our own devices, the chips that are in them, the data centers, etc)

• moving to renewable energy is difficult

• rebond effect> more efficency in renewables results in more demand for energy

• • is this also because people think that things are getting better so they stop being efficient?

• Laura presents different strategies a person can reduce energy usage of digital technologies

• broadcasting vs narrowcasting

• Organisations

• Media platforms

• Manufacturers

• Service providers

• many of the addictive practices are also high-energy practices (like endless streaming, going to the next video automatically, etc)

• Undesign principle

• reducing energy use is not in the economic interest of manufacturers

• charging for digital services including environmental footprint

• collapse informatics, planning for future scarcity

• Data center operators

• the claims of reusable energy use by big companies (like Apple) actually push other smaller users off the grid (we see this happening in NL as well with the big data centers, I believe)

• Federal governments

• tax on service providers is always offset to consumers

• default to low resolution by legistlation

• do not subsidize private coorporations

• intervene in addictive design

• International governments

• document shared in chat

• SFMF participants are interested in doing a creative practice that is also an energy saving practice *

• algorithmic colonializm event from The Hmm - free internet only through facebook app - capturing people through scarcity

• if you dont familiarize yourself with the services like AI you are going to miss out on the future. this is the emotion they are capitalizing on

• Dotcom infrastructure > video streaming > machine learning

• Datacenters could be hotter but they need to accomodate human workers who are present very rarely. (something we learned with The Hmm during our data center tour)

Reading recommendation from Laura:

Anne Pasek - Getting into fights with Data Centers : https://emmlab.info/Resources_page/Data%20Center%20Fights_digital.pdf

**Radek’s presentation and part: **

• Ireland is considered datacenter capital of europe

• Datacenter 27 percent of Irelands energy usage by 2030

• Peat extraction > eco energy parks > big tech building data centers to decarbonize

• Radek proposes that if we change things on the ground, of the individual user, then it will also change things on a higher level

• semiconductors need to be at absolute 0 so they need constant cooling

• small file solutions for makers presentation

• smallfile.ca/make

• using “obsolete” media and analogue processes (i.e. anthotype)

• stills and audio, Chris Marker

• Pixel sorting in Processing, stitching still frames in Quicktime

• Animation

• Gifs

• Compression algorithms

• Datamoshing (Avidemux)

• Camerawork for compression

• Executables (demoscene)

• OBS screen decoder (obsproject.com)

• small file media stack

• -thinking beyond indvidual files and into sustainable systems

• Autumn in the City 8MM camera, Inanna Cusi and Rashi Sethi

• La jetee, Chris Marker, sequence of still photos - effective way of storytelling

• Krystof Zanussi, Structure of Crystal, cutaway scenes

• looking at how small file techniques are inserted in traditional film maker historically

• pixel sorting

• today focusing on compression and datamoshing

• Radek uses a VLC player for previewing files because it supports a wide range of files like .mkv

• HandBrake: open source video transcoder

• • crunching the final file at the end of the artistic process into a small file
• • its very efficient at compressing
• • can be used for aristic effects
• • frontend for FFmpeg > encoder also used by youtube under the hood
• • FFmpeg Batch AV converted

• Datamoshing: manipulation of keyframes of a video

• two types of compression

• • intra frame compression - frame by frame
• • inter frame compression - from relative movement between the frames, time based predictions on sequences of frames
  • • what is encoded is difference between frames
  • • movement predicition
  • • with datamoshing you are intervening in the movement compensation process

• We begin with a hands-on workshop on Handbrake

• • using handbrake, we change the resolution limit to 480p NTSC SD from 1080p
• • another way is to change the image quality (Laura added a note in the chat: Note there are apps that record super low bitrate video on your phone:

iPhone: 0.3mp Camera (on iOS, uses H.264)

Android: Low Camera—Low Resolution)

• • in the quality selector on handbrake (in the video section) increase the RF (ratio factor) to a higher setting
  • • Radek likes to use 32
  • • a high ratio factor creates boxy pixels usually

• the choices we make here depend on our own artistic practices and the aesthetics we want to achieve

• in the audio section

• • we can have a mono track
• • reducing bitrate
• • we change the audio codec to mp3
• • change mixdown to mono and bitrate to 32

• important to change the name of the file and save as you go along, so you can see what the different changes do to your file size

• they encourage artists to also indicate the processing time

• we compressed from 65mb to just under a mb

• its effective because the movement in the film is just a small part of the image

• Hotel Moldova film by Radek with little movement

• In the Dimensions section you can do a custom resolution. Radek did 32 x 32 pixels during this presentation

• Filters can also be used to experiment

• code offers deeper level of configurations

Laura’s summary:

            1. retain quality:

decrease ratio factor on Quality slider. At 51 (low end), get boxy pixels. 32 is optimal, no compression artifacts.

        change audio parameters: mono, reduce bitrate from 160 to 32, change codec to MP3, keep sample rate. Retains audio quality.

        like baking a cake 

        I did this and changed file from 59 MB to 3.9 MB with little quality loss

        2. artistic effects:

        decrease number of pixels

        a. decrease ratio factor to max. 51. Can be overused. Super blurry, boxy pixels, things that didn’t move are moving liquidly, looks melted

        b. dimensions: resolution limit: custom: 32x32. Metallic-looking abstraction

        c. filters:

sharpen: unsharp: very strong: variables. Makes image crisper but increases file size. No interesting results so far

• Avidemux for datamoshing

• one of the issues is that it crashes a lot and on a Mac you have to change the security settings to access it

• Mpeg 4 ASP codec is the first choice then press configure

• GOP size: area bound by keyframes

• increase GOP to 500 to artifically increase the number of keyframes to make the software find keyframes where normally there would be none

• In the video filter manager there are different effects that can be used

• Go to Rotate and rotate by 90 degress

• the encoding is not as quick as handbrake and sometimes it crashes

• filters are not intrinsic to the image, its information thats coming from outside, Radek feels it betrays the artistic approach

• Radek then saved the file and then re-opened it in Avidemux

• • change video output to ‘copy’
• • he was going through the keyframes an deleting them
• • when you click the upwards arrow it goes to the next keyframe
• • once you are at the keyframe, press the ‘A’ button in the panel
• • then click the right arrow, and then click ‘B’ which sets the play mark
• • then click the ‘x’ button to delete the seleciton
• • you do this over and over until all the keyframes are gone
• • when an error message pops up that is okay, you can continue
• • then save the file

• Then he opens the saved file in the VLC player

• in this case the experiment didnt lead to a result, we need to try with a different codec, this is part of the process and can take a long time

• Now he opens the file again and picks the HEVC (x265) codec

• in the ‘frame’ tab

• changes the GOP size to a maximum of 1000

• the iframe threshold wa decreased from 40 to 10

• Radek also rotated the image to 90 degrees

Laura’s summary:

Crashes. May need to change security settings on Mac, hard if you don’t own the computer . Finicky

1. top L icon, open video

video output: MPEG4 ASP (xvid 4). Configure: change number of frames (increase Gop size to max value, 500, to artificially increase # of keyframes). Can decrease number of B frames to 0. Can change quantizer. Press OK

filters: interlacing, colors, sharpness (increased for Mountain Soul Assemblages), ASCII view, rotate, artistic, etc. Not intrinsic compression artifacts but add-ons, heavy-handed, betrays compression aesthetics (modernist faithfulness to medium)

Save (top L second icon)

Next, open saved image and delete keyframes:

use arrows at bottom of window: go to next keyframe, delete (press A button, start marker, press B. button. Press red X button to delete ‘Video will be corrupted. Proceed?” Yes)) repeat. Think first you identify keyframes, then mark and delete them. Shortcut: hit up arrow to go to next keyframe

for this video not much happened

1. Try with another codec: HEVC

Configure: Change Gop Size to max of 1000, decrease I-frame Threshold to 10. Tricks program into identifying more keyframe.

Save

delete keyframes as before, save.

Beautiful creeping textures. Video creates movement where there was none. File size reduced?

• compared to handbrake this is a longer process

• a good workflow suggestion from Radek is to create a datamoshed image in avidmex and then decrease the size in handbrake

• “datamoshing is precisely about the corruption of the image” - Radek

• making visible a memory or shadow of the keyframes while removing the keyframes themselves which are supposed to be the porper representation of the video

• Jan Kees suggests that another option is to just keep the keyframes

• comment from Laura: One thing I love about both compression and datamoshing is they create a kind of sympathetic movement across the surface of the image, suggesting everything is connected

Future question for Radek:

• the ecological impact of the making of the work vs the final file size…what are we looking at?

• Laura: they asked people to include their processing time

• reuse of existing material instead of shooting new films

• they encourage people to begin with a small file

• shooting with an old device

• Jan Kees when you go down in frame rate you might want to be in control of what those keyframes are that are kept

• it’s difficult to do live while also having low latency

Part 2 -

Presentation and workshop from Laura and Joni on small file production methods.

Notes and documentation from workshop / presentation

Participants: