01 — Title + idea

Pathway A / Net-based Creative Coding

Beat
by Beat.

Compose a rhythm.
Watch it leave a trace.

A small browser instrument where a 16-step drum pattern generates a live visual composition.

Oliver LiProject proposal · not a finished program
Speaker notes

My proposed project is called Beat by Beat. I am choosing Pathway A, net-based creative coding. The idea is a small browser instrument that lets a visitor compose a clear drum rhythm and see it become a visual pattern. Instead of following an existing song or trying to hit notes correctly, the visitor chooses where sounds occur in a repeating loop. Each sound also creates a shape. This presentation describes the project I plan to make, rather than a completed musical program.

02 — Inspiration

Three ideas to
bring together.

01

Roland TR-808

A sequence divided into 16 steps.

My takeaway: make rhythm visible as discrete choices, rather than requiring precise live performance.

Official specifications ↗
02

Chrome Music Lab

Hands-on music experiments in the browser.

My takeaway: make the first interaction simple enough to invite experimentation without a tutorial.

Explore the project ↗
03

Patatap — Jono & Lullatone

An action produces both sound and a moving shape.

My takeaway: connect sound and image directly, but use a repeating grid rather than free keyboard performance. Avoid full-screen flashes.

Visit Patatap · flashing-image warning ↗
Speaker notes

I have selected three starting references for different reasons. The TR-808 gives me a structural idea: a rhythm can be arranged through sixteen steps. I am borrowing that principle, not recreating its interface or copying its sounds. Chrome Music Lab suggests that a browser can offer a welcoming place to experiment with music. Patatap connects an action with both sound and animation. I want to combine these ideas: an accessible rhythm grid with a direct visual response. My difference is that the pattern repeats, so visitors can hear and see how a small change affects the whole composition. These references are starting points for further testing, not evidence that my own system already works.

03 — Creative motivation

From listening
to making.

I want to create a rhythm myself,
not only watch a visualizer respond to music.

My starting point

I am new to coding and interactive production. My current interest is clear drum beats that are easy to combine with visual effects.

The creative question

How can a simple repeated structure become a personal audiovisual composition through small choices?

Motivation draft based on my current interests; to refine through making and feedback.

Speaker notes

My starting point is an interest in music, rhythm and the relationship between drum beats and visual effects. I want the visitor to create music through their own choices, rather than just play a song and watch an animation. I am new to programming, so the project needs a clear and limited structure. The grid makes that limitation useful: there are only a few sounds and positions, but they can produce different combinations. I want to explore whether repetition and small changes can become a form of personal expression. This is my current motivation, which I expect to refine through making.

04 — Artistic plan (Pathway)

One loop.
Three visual voices.

Pathway A · 3 drum layers · 16 steps · fixed tempo

Concept sketch only — this grid does not play sound.

Kick → expanding ring

Weight and pulse.

Snare → short line

Accent and interruption.

Hi-hat → small points

Texture and subdivision.

Click a cell → store the pattern → play each active step → trigger its visual trace → fade the trace.

Speaker notes

The first version will have three rows: kick, snare and hi-hat. Each row has sixteen positions in a repeating bar. The visitor clicks a position to turn it on or off, then starts the loop. A moving indicator shows which step is playing. The illustration here is only a concept sketch, not a working prototype. When a kick actually sounds, it creates an expanding ring. A snare creates a short line, and a hi-hat creates small points. These shapes fade slowly enough to leave a temporary trace, so the rhythm creates a composition over time. Editing a step changes both the sound and the image. The artistic intention is to make the structure of rhythm visible and changeable. It is not a game with scores or a correct answer. The first version stays at one tempo and uses restrained movement rather than full-screen flashing.

05 — Tools and production

Build the clock.
Then the composition.

Planned tools

p5.js ↗ for the grid and visual canvas.
p5.sound / Web Audio ↗ for sound and timing.
Local HTML, CSS and JavaScript in my Living Studio.

Use self-made or clearly licensed drum sounds, and record their sources. Exact sounds are not selected yet.

First-version boundary

3 sounds, 16 steps, 1 tempo.
Play, pause, clear and volume control.
No accounts, song upload, AI composition, scoring or export.

  1. 01

    One sound, one reliable loop

    Test timing before adding visual complexity.

  2. 02

    Three editable layers

    Check toggling, pause/resume, silence and clearing.

  3. 03

    Visual mapping and user testing

    Check audio–image alignment; test with a peer and in a second browser.

Speaker notes

I plan to use p5.js for the interface and drawing, with p5.sound and Web Audio for playback and timing. I will start with one sound and one reliable loop. Audio needs its own clock; I should not assume that drawing frames will keep accurate musical time. Next, I will add the three editable rows and basic controls. Only then will I connect the sounds to the visual traces. I will use self-made or clearly licensed sounds and document their sources. My tests will include changing steps while the loop runs, clearing everything, pausing and restarting, and checking whether the image follows the sound. I will also ask a peer to try it and test another browser. The first milestone is a stable loop, not a feature-rich music app.

06 — AI & Questions

AI helps me build.
I decide what it means.

What I will ask AI to do

Explain the pattern array and audio clock.
Generate and revise small functions.
Help diagnose bugs and compare implementations.

What remains my responsibility

Choose and credit sounds.
Decide the visual mappings.
Test generated code and explain its logic.
Record decisions, failures and next actions.

How can a limited rhythm grid support personal expression without making every composition feel the same?

Feedback I need: do the three visual mappings make the rhythm easier to understand, or distract from listening?

Speaker notes

AI will support my learning, but it will not be the composer or the artistic decision-maker. I will ask it to explain the pattern data, generate small functions and help investigate bugs. I need to be able to explain how a clicked cell changes the pattern, how the loop decides which sound to play, and how that event creates a visual trace. I will keep a record of suggestions I accept or reject, failed attempts and the tests I run. My main emerging question is how a limited grid can support personal expression without making every composition feel the same. I would like feedback on whether the visual mappings clarify the rhythm or distract from listening. My next action is to test a single drum loop.

Back to the Living Studio ↗