chapter

WP3:_SMALL_FILE_PUBLISHING_Internal_pad

**Notes Pad - WP3_Internal Work Session 1 **

This is the pad for our first internal work session. This session will be split into two parts:

Part 1 with Radek and Laura: Tuesday May 26 from 16-19h (Amsterdam time)

Part 2 with Joni and Laura: **Tuesday June 16 from 16-19h (Amsterdam time) **

The sessions will take place online in Jitsi here: https://meet.greenhost.net/connecting-otherwise

Part 1 -

Presentation and workshop from Radek and Laura of the environmental impact of streaming and the research SFMF have done around this topic, as well as a more hands-on workshop on different methods and options for compression, and the choices that are made (i.e. learning about data moshing techniques and open source programs that can be used for making small files). Following this part we then have time to try out the different methods Radek showed, on our own time, and document them, so we can discuss them together in Part 2.

Notes and documentation from workshop / presentation

**Participants: **

Laura

Mark

JK

Juliette

Radek

Margarita

Introduction of Jan Kees

• Composer by training, Tv broadcast via internet

• Lot of radio, Icecast, Liquidsoap

• Interested in broadcast and livestreaming regarding small file

• reduction of latency, gap between video meeting immediacy and live broadcast

• WebRTC and even faster stuff by the guz who developed VLC

• very low frame rate video broadcast, audio is immediate thats enough to perceive well

• PUB rietveld

**Laura presentation / introduction on the different actors involved in the carbon emissions of streaming **

• Laura shares screen, checking in from Texas

• best practices, decreasing carbon emissions of information and communication technologies

• Laura is calculating that global greenhouse emissions from streaming are 5.7% (this includes our own devices, the chips that are in them, the data centers, etc)

• moving to renewable energy is difficult

• rebond effect> more efficency in renewables results in more demand for energy

• • is this also because people think that things are getting better so they stop being efficient?

• Laura presents different strategies a person can reduce energy usage of digital technologies

• broadcasting vs narrowcasting

• Organisations

• Media platforms

• Manufacturers

• Service providers

• many of the addictive practices are also high-energy practices (like endless streaming, going to the next video automatically, etc)

• Undesign principle

• reducing energy use is not in the economic interest of manufacturers

• charging for digital services including environmental footprint

• collapse informatics, planning for future scarcity

• Data center operators

• the claims of reusable energy use by big companies (like Apple) actually push other smaller users off the grid (we see this happening in NL as well with the big data centers, I believe)

• Federal governments

• tax on service providers is always offset to consumers

• default to low resolution by legistlation

• do not subsidize private coorporations

• intervene in addictive design

• International governments

• document shared in chat

• SFMF participants are interested in doing a creative practice that is also an energy saving practice *

• algorithmic colonializm event from The Hmm - free internet only through facebook app - capturing people through scarcity

• if you dont familiarize yourself with the services like AI you are going to miss out on the future. this is the emotion they are capitalizing on

• Dotcom infrastructure > video streaming > machine learning

• Datacenters could be hotter but they need to accomodate human workers who are present very rarely. (something we learned with The Hmm during our data center tour)

Reading recommendation from Laura:

Anne Pasek - Getting into fights with Data Centers : https://emmlab.info/Resources_page/Data%20Center%20Fights_digital.pdf

**Radek’s presentation and part: **

• Ireland is considered datacenter capital of europe

• Datacenter 27 percent of Irelands energy usage by 2030

• Peat extraction > eco energy parks > big tech building data centers to decarbonize

• Radek proposes that if we change things on the ground, of the individual user, then it will also change things on a higher level

• semiconductors need to be at absolute 0 so they need constant cooling

• small file solutions for makers presentation

• smallfile.ca/make

• using “obsolete” media and analogue processes (i.e. anthotype)

• stills and audio, Chris Marker

• Pixel sorting in Processing, stitching still frames in Quicktime

• Animation

• Gifs

• Compression algorithms

• Datamoshing (Avidemux)

• Camerawork for compression

• Executables (demoscene)

• OBS screen decoder (obsproject.com)

• small file media stack

• -thinking beyond indvidual files and into sustainable systems

• Autumn in the City 8MM camera, Inanna Cusi and Rashi Sethi

• La jetee, Chris Marker, sequence of still photos - effective way of storytelling

• Krystof Zanussi, Structure of Crystal, cutaway scenes

• looking at how small file techniques are inserted in traditional film maker historically

• pixel sorting

• today focusing on compression and datamoshing

• Radek uses a VLC player for previewing files because it supports a wide range of files like .mkv

• HandBrake: open source video transcoder

• • crunching the final file at the end of the artistic process into a small file
• • its very efficient at compressing
• • can be used for aristic effects
• • frontend for FFmpeg > encoder also used by youtube under the hood
• • FFmpeg Batch AV converted

• Datamoshing: manipulation of keyframes of a video

• two types of compression

• • intra frame compression - frame by frame
• • inter frame compression - from relative movement between the frames, time based predictions on sequences of frames
  • • what is encoded is difference between frames
  • • movement predicition
  • • with datamoshing you are intervening in the movement compensation process

• We begin with a hands-on workshop on Handbrake

• • using handbrake, we change the resolution limit to 480p NTSC SD from 1080p
• • another way is to change the image quality (Laura added a note in the chat: Note there are apps that record super low bitrate video on your phone:

iPhone: 0.3mp Camera (on iOS, uses H.264)

Android: Low Camera—Low Resolution)

• • in the quality selector on handbrake (in the video section) increase the RF (ratio factor) to a higher setting
  • • Radek likes to use 32
  • • a high ratio factor creates boxy pixels usually

• the choices we make here depend on our own artistic practices and the aesthetics we want to achieve

• in the audio section

• • we can have a mono track
• • reducing bitrate
• • we change the audio codec to mp3
• • change mixdown to mono and bitrate to 32

• important to change the name of the file and save as you go along, so you can see what the different changes do to your file size

• they encourage artists to also indicate the processing time

• we compressed from 65mb to just under a mb

• its effective because the movement in the film is just a small part of the image

• Hotel Moldova film by Radek with little movement

• In the Dimensions section you can do a custom resolution. Radek did 32 x 32 pixels during this presentation

• Filters can also be used to experiment

• code offers deeper level of configurations

Laura’s summary:

            1. retain quality:

decrease ratio factor on Quality slider. At 51 (low end), get boxy pixels. 32 is optimal, no compression artifacts.

        change audio parameters: mono, reduce bitrate from 160 to 32, change codec to MP3, keep sample rate. Retains audio quality.

        like baking a cake 

        I did this and changed file from 59 MB to 3.9 MB with little quality loss

        2. artistic effects:

        decrease number of pixels

        a. decrease ratio factor to max. 51. Can be overused. Super blurry, boxy pixels, things that didn’t move are moving liquidly, looks melted

        b. dimensions: resolution limit: custom: 32x32. Metallic-looking abstraction

        c. filters:

sharpen: unsharp: very strong: variables. Makes image crisper but increases file size. No interesting results so far

• Avidemux for datamoshing

• one of the issues is that it crashes a lot and on a Mac you have to change the security settings to access it

• Mpeg 4 ASP codec is the first choice then press configure

• GOP size: area bound by keyframes

• increase GOP to 500 to artifically increase the number of keyframes to make the software find keyframes where normally there would be none

• In the video filter manager there are different effects that can be used

• Go to Rotate and rotate by 90 degress

• the encoding is not as quick as handbrake and sometimes it crashes

• filters are not intrinsic to the image, its information thats coming from outside, Radek feels it betrays the artistic approach

• Radek then saved the file and then re-opened it in Avidemux

• • change video output to ‘copy’
• • he was going through the keyframes an deleting them
• • when you click the upwards arrow it goes to the next keyframe
• • once you are at the keyframe, press the ‘A’ button in the panel
• • then click the right arrow, and then click ‘B’ which sets the play mark
• • then click the ‘x’ button to delete the seleciton
• • you do this over and over until all the keyframes are gone
• • when an error message pops up that is okay, you can continue
• • then save the file

• Then he opens the saved file in the VLC player

• in this case the experiment didnt lead to a result, we need to try with a different codec, this is part of the process and can take a long time

• Now he opens the file again and picks the HEVC (x265) codec

• in the ‘frame’ tab

• changes the GOP size to a maximum of 1000

• the iframe threshold wa decreased from 40 to 10

• Radek also rotated the image to 90 degrees

Laura’s summary:

Crashes. May need to change security settings on Mac, hard if you don’t own the computer . Finicky

1. top L icon, open video

video output: MPEG4 ASP (xvid 4). Configure: change number of frames (increase Gop size to max value, 500, to artificially increase # of keyframes). Can decrease number of B frames to 0. Can change quantizer. Press OK

filters: interlacing, colors, sharpness (increased for Mountain Soul Assemblages), ASCII view, rotate, artistic, etc. Not intrinsic compression artifacts but add-ons, heavy-handed, betrays compression aesthetics (modernist faithfulness to medium)

Save (top L second icon)

Next, open saved image and delete keyframes:

use arrows at bottom of window: go to next keyframe, delete (press A button, start marker, press B. button. Press red X button to delete ‘Video will be corrupted. Proceed?” Yes)) repeat. Think first you identify keyframes, then mark and delete them. Shortcut: hit up arrow to go to next keyframe

for this video not much happened

1. Try with another codec: HEVC

Configure: Change Gop Size to max of 1000, decrease I-frame Threshold to 10. Tricks program into identifying more keyframe.

Save

delete keyframes as before, save.

Beautiful creeping textures. Video creates movement where there was none. File size reduced?

• compared to handbrake this is a longer process

• a good workflow suggestion from Radek is to create a datamoshed image in avidmex and then decrease the size in handbrake

• “datamoshing is precisely about the corruption of the image” - Radek

• making visible a memory or shadow of the keyframes while removing the keyframes themselves which are supposed to be the porper representation of the video

• Jan Kees suggests that another option is to just keep the keyframes

• comment from Laura: One thing I love about both compression and datamoshing is they create a kind of sympathetic movement across the surface of the image, suggesting everything is connected

Future question for Radek:

• the ecological impact of the making of the work vs the final file size…what are we looking at?

• Laura: they asked people to include their processing time

• reuse of existing material instead of shooting new films

• they encourage people to begin with a small file

• shooting with an old device

• Jan Kees when you go down in frame rate you might want to be in control of what those keyframes are that are kept

• it’s difficult to do live while also having low latency

**MARGARITA tool testing notes **

Handbrake

• using handbrake, I compressed one 22mb video to 659kb
• it was more pixelated but still not bad for such a big drop
• it had water and was quite blue, and I wonder if that made the pixels stand out more
• I changed the framerate to ‘constant framerate’ and it didn’t alter anything with the size
• I then tried a second video where it was a snow storm, and then the pixelation kind of worked with the energy of the video, a kind of disorted scene

• I then tried it again with the snow storm video but with an RF quality setting of 13,5, but then only reduced the video down to 13mb Avidemux

• I followed the steps but when I tried to save the file I had one error after another and nothing really happened. One of the errors was that it was not saved correctly

• I could not resolve the errors :( ffmpeg via homebrew via antigravity (sorry!)

• got things down to about 3mb

• Both options successfully removed the massive unneeded spatial audio/metadata tracks while preserving high visual quality at 1080p.
• the tool did the following:
  • Analyzed the original video (IMG_2488.mp4, 22.5 MB) and identified unnecessary spatial audio and metadata streams.
  • Compressed with H.265 (HEVC) to create compressed_hevc.mp4 (3.07 MB, 86.4% size reduction, highly compatible).
  • Compressed with AV1 to create compressed_av1.mkv (3.12 MB, 86.1% size reduction).
• I asked how much energy it used in the whole process:
  • Total Energy Consumed:
  • Combined, the entire process (AI thinking + local rendering) used approximately 5 to 15 Watt-hours of electricity.
  • To put that in perspective, the entire operation used less energy than it takes to boil a single cup of water (which takes about 15–20 Wh)!

Part 2 -

Presentation and workshop from Laura and Joni on small file production methods.

Notes and documentation from workshop / presentation

Participants: Margarita, Joni, Laura, Alix, Juliette, Meis, Jan-Kees, Mark

https://smallfile.ca/tinydreams/

**Reflections on using tools from previous session. **

• Laura says that finding the sweet spot between too little or too much compression makes it better.
  • tip from Laura via Radek about the sweet spot: decrease ratio factor on Quality slider. At 51 (low end), get boxy pixels. 32 is optimal, no compression artifacts.
  • because compression tends to muddy the colours, use your editing programs to increase the saturation
• Margarita was having tehcnical problems with the software crashing for Avidemux
• Margarita asked a vibecoding tool to make a small file. It used ffmpeg and revealed that used the energy equalent of boiling a cup of water.
• Juliette: the threshold between it becoming pixelated and more of an aesthetic choice is tough to figure out
  • what are the parameters that are acceptable to our eye
  • using datamoshing to create transitions between video segments
  • are we having a meeting online, is what I thought ? (jk) - Yes, its here: https://meet.greenhost.net/connecting-otherwise We can see you
• Jan Kees experimented with ffmpeg
  • using command line with ffmpeg and H265 for compression
  • he wants to try making a web interface for this, so it’s more accessible
• Laura
  • in general they are interested in making files as small as possible and still of a good quality / usable
    • tiny files that can still effectively communicate
  • they are working with people in Sudan, Iran, Kashmir, etc.
  • Kashmir activists need to communicate on 2G network
• Jan Kees wants to work on creating some tools and sharing them

• Meis workshop manager Rietveld editing studios

  • interested in small files and recording things in a lower resolution and in different formats
  • questioning that students are often creating huge files **Joni workshop segment **

• Vancouver. Working with Laura, SFMF 7 years. Freelance webdeveloper and work with folks that run experimental film festivals.

• Interest is learning towards making small file media appealing.
• Laura: Joni’s innovations like pixel squisher to allow people to make small file videos on their phones.
• https://smallfile.ca/tinydreams -> we are following this website along
• environmental and accesibility stakes of web bloat
• Laura says that it’s pretty sure that we will soon have a drop in electricity excess and inability to convert to renewbles (peak oil) small files are part of that movement
• 1.44mb per second for video but no standard for web
• when doing small file work we tend to look backwards rejecting new technology
• 5x growth in bloat in 15 years
• forgoing advances that emphasize comfort over efficiency: high def media, legible images, non system fonts, global cdn access etc..
• Wikipedia is quite light weight
• small file coding approaches are more beginner friendly
• simple websites are more archivable
• benefits: accessibilty, performance, sustainability, longevity
• On the website for the presentation, there are example of early internet art that is small plus a series of websites that are really small
• https://wttr.in/ functional small file website
• https://jschinkel.com/spiritscripts website made by Joni meant to be a homepage. Interactive site made with small file approach in mind
  • the website is meant to mimic some of the early text based video games
• Jan Kees: simple animations can be done with CSS
• Alternative image formats for small file using CSS, ASCII, etc
• 2D video games use spritesheets of characters. Use CSS to animate
• Juliette shared a website that uses a spreadsheet: https://moubarak.eu/#welcome
  • the PUB website uses a visual grid as a calendar (done in css) : https://pub.sandberg.nl/
• Alix used block characters to create images on the Connecting Otherwise project
• Before permacomputing was a concept, the demoscene was already doing this. The demoscene is an international computer art subculture dedicated to making self-contained audiovisual programs - demos - that run from a single executable file with no assets. Competitions are structured by strict size limits: 64k intros (65,536 bytes), 4k intros (4,096 bytes), 1k intros (1,024 bytes), and smaller. A winning 4k intro might contain minutes of real-time 3D animation with procedurally generated music, produced entirely from 4,096 bytes of machine code - no images, no audio files, everything synthesized from math at runtime. (from Joni’s presentation)

• **From Joni’s presentation: **

  • Frugal, salvage, and collapse computing (100 Rabbits / Devine Lu Linvega)

  • The design collective 100 Rabbits - Rek and Devine Lu Linvega, who live and work on a 1982 sailboat - have developed one of the most rigorous practical frameworks for small-footprint computing. Powered by two 100w solar panels, operating from anchorages with intermittent or no internet, they’ve built their entire software stack from scratch: programming languages, notation tools, sprite editors, music sequencers - all running on 10-year-old laptops, all offline-first, all small.
    • Their framework describes three stages of adaptation:
      • Frugal computing: treating computational resources as finite. Using only what you need, when you need it, while resources are still available - learning to work within constraints before you’re forced to.
      • Salvage computing: limited to already-produced hardware and software. No new production; only what exists.
      • Collapse computing: working with what survives the failure of industrial supply chains and network infrastructure.
  • Small file web sits squarely in frugal computing. The practice is to internalize resource constraint now - not because infrastructure is failing, but because the habit of frugality changes what you make and how you think about it. Their tools, essays, and talks are all documented at 100r.co.
  • Jan Kees: instead of a raspberry pi we can use an ESP 32 (42?) as it also has a low power setting
  • Important: folk web practice can create problems with technical accessibility
  • ASCiI art not readable by screen readers
  • Aggressive image compression can destroy legibility for people with low vision.
  • Low contrast aesthetics - muted palettes, grey-on-grey, light text on light backgrounds, are a frequent feature of folk web and minimal design traditions and a common WCAG failure.
  • Pixel art rendered with image-rendering: pixelated at small sizes can be illegible to people with low vision.
  • WCAG web accessibility standard
  • surveillance dimension: Plain HTML pages with no external requests, no analytics scripts, no CDN dependencies, and no third-party embeds leave a smaller tracking footprint than pages that phone home to a dozen domains on every load. This matters in environments where what someone reads online can be monitored. A self-contained page that loads from a single server request and makes no further network calls provides structural privacy properties no privacy policy can match.
  • Joni has a section on the site that maps out what his ideal small file website looks like
  • Jan Kees shared a one-file CMS: https://onefilecms.com/
  • Jan Kees shared some nice websites:
  • https://ascii.theater/
  • https://asciinema.org/
  • Websites that we explored to see their size and how they are working:
  • http://www.mathcats.com/
  • https://rectangles.app/
  • https://spork.org/
  • http://towel.blinkenlights.nl/ (needs to open in terminal)?
  • https://www.seriss.com/people/erco/fovicks/
  • https://fiftythousandnames.org/
  • original fonts
  • serif
  • san serif
  • monospace
  • cursive
  • fantasy **Some Rhetorical questions from Joni to think about: **

On the standard

   * The 1.44 MB/minute metric works for video because it has duration. What would the equivalent be for a webpage? Transfer size? Number of requests? Energy per visit? Carbon per page load? Do any of those feel right?
   * Is "small file" a technical definition or an ethical one? Can a site be technically small but philosophically bloated?
   * If a site is beautiful and costs 3MB to load, is that justified? What if it's ugly and costs 3MB? What if it's beautiful and costs 30KB?

On usefulness and application

   * The folk web tradition celebrates useless sites… one joke, one horse, one colour. The political accessibility argument says small file matters because it works under constrained conditions. Are those the same project?
   * Who are you making a small file site for? Does the answer change what "small" means?
   * Is optimising for constrained infrastructure an act of solidarity, or just good practice? Does the distinction matter?

On the web as a medium

   * View-source used to be how people learned to make websites. It assumed a culture of legibility and sharing. Do we want that back? What would it take?
   * A website that works offline, on a USB drive, over a mesh network, or after the host goes under is more durable than one that depends on a platform. Is durability an aesthetic value as well as a practical one?
   * What does it mean that the most "progressive" web development practices (heavy frameworks, server-side rendering, build pipelines) produce the least accessible and least durable output?

On aesthetics and politics

   * The folk web looks like the early internet. Is that nostalgia, or is it the most honest visual language for this kind of work?
   * Dithered images, system fonts, visible compression… these aesthetics emerged from constraints. When you choose them intentionally, is that something different?
   * Low Tech Magazine goes offline when the sun doesn't shine. That unavailability is designed in. What else could be designed to make its constraints visible rather than hiding them?

Open-eneded questions

   * What would you make if file size was the only constraint?
   * What's the smallest thing that could still be a website?