The Myth of High Resolution

Graphic artists have been fed this myth that they need to work at higher and higher pixel resolutions.  That it is vital to their work to do so.

Graphic artists are shaking their heads in agreement (with the second part, not the first). Yes, exactly. Give us more resolution. So that we can work at higher and higher resolutions.

Computer manufacturers are also shaking their heads in agreement. Yes, exactly. These people need to buy new equipment from us every year that displays at higher and higher resolutions.  They need to work with raw pixels at higher and higher resolutions.  They need to buy new stuff (from us of course).

Now we know why the hardware manufacturers are pushing this narrative.  Because they want your money.  They want you to buy new stuff every year (they would prefer every 6 months).

And computational power.  Yeah, sure.  I get that totally. Come on hardware manufacturers, give me more computational power in your systems so i can use it all for some amazing new algorithm that needs all of it and more.  Think of what we could do with that.

But more and more resolution?  And then having to actually work at that increased resolution?  Why exactly?

Graphic artists speak up in a loud voice.  But we need to work at those higher and higher resolutions so that our prints of our work look perfect in every detail. No mater if the viewer is across the room looking at it through coke bottle eye glasses, or up real close up with a loop magnifying lens (different kind of coke bottle lens) so they can see all of the details.

Yeah, i get that.  You want your digital prints to look good.  Especially when you print them wall sized in the art gallery of your dreams (i'm sure there are a few, like that place in Lahania).

Graphic artists again insist.  We need that high resolution because the stock place (substitute the magazine, ad agency, special effects house, etc) that is buying our art insists.

Sure, i totally get that.  Your final work output has to be a certain resolution or the people buying it will not pay you money for it. Absolutely, it's what the client wants.

Video artists speak up. Buddy, we needed 4K because we needed to build content to support future 4K video streaming services (even though most people watch them with a streaming bandwidth that probably doesn't even reliably support SD resolution). And now we need 8K because everyone who buys our stuff wants it archived for the future, so 4K just doesn't cut it.

Alright, i understand that even though Netflix can't even support SD streaming resolution without grinding to a halt where i live, you need to sell your content to them at 8K because that is what they want, or more likely your boss wants it at 8K so he can resell it to them later at a higher res, sure. It's all about expectations.  They have to be met.

They have to be met in the final end stage.

But working resolution?  Why does that have to be the same.

And it's fascinating to see Nvidia push this whole concept of neural net based image interpolation converting a much lower resolution real time ray-traced graphics render into a much higher resolution 4K video display frame for gamers to enjoy.

And this leads us into the whole myth of high resolution that digital artists have been fed.  Because it's only true for the final end stage.  The part where you put together whatever final thing you are blasting off to the client, or to the ink jet printing place, or the Netflix streaming vault, etc.

But working resolution.  Why does that have to be high res?

But edges have to be sharp. I need to zoom way, way, way, way in to make sure that is true.

Sure, edges have to be sharp. If you want them to be.  But that zoom way, way, way in part, just to check, just to make sure? While you are working on design, on composition, on the structure?  That part seems odd.

Especially if it then requires you to work with order of magnitude or more more raw pixel information when you are doing your compositional work.  Every time you double the canvas dimensions, you 4X the amount of data.  You 4X the cpu power required, you 4X the bandwidth required, you 4X the slowdown in response time of the system.  That adds up real fast, exponentially fast.

And i'm arguing that it is totally unnecessary for the compositional part of your work. It's only an essential requirement for your final output.

Nvidia totally gets this. That is why they are restructuring the entire graphics pipeline of computer game real time graphics display to take advantage of it.

Synthetik gets it as well. We want to structure our working environment for digital artists so that they can separate the concept of interactively (or automatically, or intelligent assisted, etc) building the digital art content from the totally separate super high resolution output they might require for the final stage of their work.

Are we totally there yet in Studio Artist V5.5? No, not yet. For some things (vector effect output), sure. And we have ideas to make that more explicit for those things (like the Vectorizer) in the near term future, to aid you in this rethink of what working resolution actually means.

For other effects, re-rendering your Paint Action Sequence (PASeq) might be a way to go. Raster paint nibs on a vector bezier path, Studio Artist has offered that since V1. For some paint effects it works great, for others, not so great.  It can at times works like an edition print (subtle or not so subtle differences in output from run to run), which could be cool when viewed in a certain light, but could be frustrating in others.

So we have other plans for future but already in development internally here at Synthetik Studio Artist V6. To cover the other work scenarios with some really cool options (think all things raster in nature).

Here's the thing.  If you look at the contrast sensitivity function of the human visual system, there is a pronounced drop off at higher visual spatial frequencies.  And algorithms for digital halftoning and for image compression have taken advantage of this fact for many years, depend on it for them to work at all. Place the error generated by the system at spatial frequencies where the viewer can't perceive it.

So when you are building up the composition and structure of your digital art, that information has very little if anything to do with it's perception. Not until you get up real, real, real close, so close you are now looking at that sharp edge like it is it's own digital art canvas, one that really has nothing really  to do with the perception of the overall work of digital art (the composition, the structure, etc).

So if you can't really even see that information when you are building up the overall structure of your piece of digital art, why would you want it to get in the way of that important creative design work by hogging 4X x 4x x 4X ad infinitum of the cpu and memory bandwidth?

Something to think about.

Studio Artist V5.5 - What is it and Where is it Headed

 StudioArtist V5.5.0 is locked down and done at this point in time.  The installer packaging and web site stuff is still being finalized (that needs to happen before it hits the online store), but the actual 5.5.0 code is done at this point.  Which is pretty liberating (at least for me), because we've already moved on and begun work on 5.5.1.

So what is Studio Artist anyway?

It's a digital art program with a 20 year long history at this point. When Studio Artist V1 was released at MacWorld New York in 1999 (is that a long time ago or yesterday, i get confused), it was a pretty radical rethink of what a digital paint program could be.  It was not PhotoShop, it was not Painter, it was not Illustrator, it was its own thing.  With its own internal structure, its own way of working, its own way of thinking about how to work.

It allowed you to work with still images or video, merging those 2 very separate application categories at the time into one unified system. It introduced the concept of a 'source' and a working 'canvas', again a very unique distinction at the time.  It introduced the concept of incorporating a human visual model (based on research into how the human visual cortex works at the neural level) into a graphics program for digital artists.  It incorporated conceptual ideas from music synthesis, music synthesizers, and digital audio software, and then incorporated them into a digital art program for visual artists.  It introduced a new hybrid raster-vector model for digital paint effects (bezier paths that define a drawing path, raster paint nibs that a relaid down on that editable vector paint path).

It also introduced what i call the first 3 levels of ways of working and interacting with digital imagery.

Level 1 is all about working at the pixel level.  Down in the basement.  Literally pushing pixels around on the canvas with an interactive stylus (wacom pen at the time) or a mouse cursor.

I would also consider the use of simple image processing effects (like an edge or blur filter) to be working at level 1 to some extent.  At least for the sake of this discussion.

Level 2 moves up a conceptual notch with intelligent automatic actions.  Intelligence based on the underlying human visual modeling built into the program.  That takes a center stage in the Studio Artist paint synthesizer with its extensive use of visual attribute modulation internally.  But intelligence that was also distributed throughout the program in a wide variety of adaptive image processing effects.  Also based on visual attribute modulation.

Studio Artist is built from the ground up on the concept that the program tries to look at a visual 'source' just like a human artist would look at that 'source'.   That internal visual representation is then made available internally throughout the program.  Visually modulating what the paint synthesizer is doing, visually modulating what adaptive image processing effects are doing.

You can think of the 'source' as a model the artist is painting, or a photograph of a scene the artist is reinterpreting in a painting.  The artist looks at the source to influence their work, but so does Studio Artist. Using its internal human visual modeling.  Trying to perceive the source in a similar way to the artist perceiving it, reacting to it, etc.

This process is not about the machine replacing the artist.  Far from it. 

Studio Artist always tries to incorporate the human artist into the automatic action creative loop. At whatever level of interaction the artist is comfortable with.  The artist can do all the work if they want to. Or the artist can manually manipulate the stylus, while Studio Artist intelligently assists in the manual drawing (intelligent assisted painting).  Or the artist can press the Action button, and Studio Artist does all of the work automatically (fully automatic action painting).  

But even in that last case of a fully automatic action, the artist is still involved in the loop, making decisions about what they like and don't like, how to proceed next, etc.

Level 3 in our taxonomy of ways of working i would define as working with scripts of actions (which could be manual or automatic actions).  Studio Artist includes a Paint Action Sequence (PASeq) Editor, that allows you to build more expansive visual effects composed of multiple actions that work in sequence.  Combining together different kinds of brush sizes, different kinds of digital media emulation, different kinds of artistic techniques (charcoal, ink pen, watercolor, water or acid wash, canvas smear etc) into a sequence of different processing steps that work together in synergy to build up a final visual effect.

Since automatic actions intelligently analyze the 'source', they can be built using just one specific image as the source when they are initially created, but then applied to an infinite variety of different source images later, while achieving the same stylistic visual effect output for those new input images as was designed using the first source.

Intelligent actions meant that you could also work at level 3 building PASeq presets in Studio Artist, and then process movies with them.  Creating dynamic visual effects in processed movie files that looked like hand painted or hand animated art styles in moving art.

That was a lot of innovation introduced in old SA V1.  Innovation that is still missing in other digital art programs 20 years later to be honest.  I say that because in every other digital art program i sit down at, you start with a blank canvas, and then you have to do all of the work.  The other programs do nothing, you have to explicitly make everything happen in them yourself.  

Here's the thing, that's a really hard process to create something from nothing.  Especially if you have not taken the extensive time and training required to develop the muscular and neural motor skills required to draw well.  Some people are very good at it, many, many more are not.

Now there's a direct analogy in the world of music composition, where in the old days a composer would compose a piece of music (as marks on paper literally), and then go into a very expensive recording studio, and tell a bunch of musicians they hired at union scale wages to play their piece of music for them, and a recording engineer you also had to pay hourly would record them playing the music, and then another person you had to pay would mix and master the recorded tracks into a finished piece of music.  And the expense and organizational complexity of all of this was really prohibitive to musical creativity.

So the home recording studio revolution created by digital audio workstation software running on personal computer hardware (a revolution i was heavily involved in created back in the day), was truly liberating for individual musicians and composers.  Because they could do the work on their kitchen table if they wanted to, and they could do all of the work if they so desired themselves.

Studio Artist was trying to do a very similar thing in the visual art and digital video world.  So that if an individual had a great idea for an animated film, they could do it all themselves on their kitchen table.  As opposed to hiring a team of animators at great expense to make that happen.

David Kaplan pursed that very vision using Studio artist to great effect, winning an award at the Sundance Film Festival for his feature length Studio Artist animated film 'Year of the Fish'.  Literally created at his kitchen table in his apartment in NYC.

And the innovation continued over the years as new versions of Studio Artist were released.  Vector effects were introduced via the Vectorizer, new vector paint options in the paint synthesizer, and vector output from some Ip Op effects.  You could output these new vector effects as resolution independent svg files, an interesting alternative to the traditional raster image file output.  

Temporal image processing effects were introduced.  These were great for processing video with time based effects, but also opened up a whole new way of working called 'stack filtering'.  Stack filtering involves taking a collection of still images, and then using that collection of individual images as the input to temporal image processing effects.  The end results can be visually amazing.

MSG (modular synthesized graphics) was introduced in SA V2, and has been extensively expanded over the years.  With over 600 individual image processing and generative modular effects that can be used to construct an infinite variety of different modular preset effects.  MSG can be totally generative (creating visual imagery from nothing but editable parameters), or it can process a source image into an effected output image, or a source can be used to modulate a generative process in some way internally.

MSG presets can also be embedded into the paint synthesizer, providing a way to modularly expand different internal paint synthesizer components (like path start generation, path shape generation, brush load processing, source brush generation).  Studio Artist V5.5 also lets you embed Ip Op and Vectorizer effects directly into the paint synthesizer (once again super-charging what you can do with paint synthesizer presets).

Movie brushes (using a movie as a paint brush) were introduced fairly early on in Studio Artist history. Movie brush capabilities have expanded over the years with image folder brushes, as well as movie and image folder background textures.  These can be incorporated along with visual attribute modulation to build photo mosaic and other visual effects built off of artist curated sets of multiple visual images loaded into a digital paintbrush.

Keyframe animation in the PASeq Timeline allows for the construction of interpolated hand painted strokes that then move dynamically over time in an animation, automatic dynamic visual transformations, morphing, warping, etc.  Bezier paths derived from automatic actions can also be embedded into single paint actions and then key-framed as well on the PASeq Timeline.

So Studio Artist has always been a generative digital art system from its very beginning.  And i want to draw a distinction between how the word 'generative art' is oftentimes used, because many artists use it to refer to digital art output from software coding systems created for artists like Processing, Open Frameworks, or perhaps mucking about in neural net Colab notebooks these days, etc.

I'm all for artists learning how to work with software code if they so desire. But my experience working with a lot of visual artists over the last 30 years indicates that many of them are not really interested in software coding.  It requires a certain level of 'left brain' analytical thinking they aren't necessarily comfortable with (especially within their working methodology which is much more intuitive or 'right brain' in nature).

These 'left brain' - 'right brain' metaphors are much over-worked (and technically incorrect) at this point in time, but i think they are still useful on some level.  We have traditionally made the distinction between 'left brain' building your set of tool presets, vs 'right brain' using your set of tool presets within the Studio Artist universe.  So you would set apart specific times to build working tools (left brain activity), and then use those tools creatively in your artwork (right brain activity).

The previous distinction between these 2 ways of working ( 'building the tools  and then 'using the tools' ) is a good lead in to where Studio Artist is heading.  Both in terms of the new features we are introducing in Studio Artist V5.5, and where we see those new features heading in the future.  Because we believe we have staked out a whole new way for digital artists to work in Studio Artist V5.5. And we are going to fully develop that out and expand the nature of what it truly means in future Studio Artist releases.

Studio Artist V5.5 introduces 2 whole new higher levels to our taxonomy of 'levels of working' for digital visual artists.

The 4th level of working involves expanding the whole concept of what the 'source' even means in a digital art program.  Traditional digital art programs are very religiously rooted to the concept of working with a single image, a single movie file.  The single digital photo you took, and you now want to enhance or effect in some way.  The single movie file that you want to effect in some way.  The empty blank canvas that you are supposed to turn into a sketch of a woman's face, or a bowl of fruit on a table.

So this new 4th level of working i call 'source abstraction'.  Abstracting and expanding the whole notion of what the source even means.  So that it is no longer tied to just a single image or movie file. It might be a representation of a collection of images, a collection of movies.  You might start to think of it as more like a data model, filled with visual attributes, all available for modulation within Studio Artist.

You can also start to think about 'style' mixing in the context of the 'source'.  Playing one abstracted 'source' off against another abstracted 'source'.

My personal viewpoint on the whole notion of what a source is, or even style for that matter, has evolved and expanded quite a bit over the years.  Just like our notion of what the source is and can do has expanded in Studio Artist V5.5.  With the new Gallery Show source options, and the new Load Style features.  Both of which are really just teasers for even cooler things to come, but still incredibly useful in their current limited forms.

It's fascinating when you start thinking about the 'source' for a piece of artwork as being a collection of images instead of a single image. It's very much like working with a database of visual attributes.  It's a very different way of thinking for most digital artists (at least that is my perception).  Different, but really expansive, liberating perhaps, definitely worth getting a grasp on, worth trying out and exploring.  It's new territory waiting to be charted out, waiting for you to find your own personal niche within it.

As i pointed out earlier, the Studio Artist universe dived into this water when we got heavily into stack filtering. But it's always existed since V1 if you really wanted to explore it, via loading a movie (which could be any old collection of images as individual frames within the movie container), and then riffing with it while painting.

I've been very heavily involved in getting up to speed on the latest developments in deep learning neural nets during this last pandemic year.  Most people are probably unaware that i did neural net research in the 90s, including some very basic work on using convolutional neural networks for learning artistic stylistic transformations on images.  Not really practical back then to be honest, so we quickly moved on to adaptive fuzzy logic systems and other things that ran a lot faster on the ancient computer hardware available at the time.

But it's a very different world now days. So it's been fascinating to take my background in all things neural net and bring it all up to date with the latest and greatest developments in the field. So i'm heavily influenced by what people have been doing with recent research areas like neural style transfer, generative models like GANs and VAE systems, etc.

And the notion of a database of visual images is very central to how these deep learning neural net systems work.  Generative imaging transformations based on the collective visual statistical properties of a collection of images. So all of that exciting new research work has heavily influenced my thinking, and is going to continue to do so in the future as we move forward.

This whole notion of source abstraction (expanding the whole notion of what a 'source' even means), is the new 4th level of working.  

And the 'build a tool' - 'use a tool' discussion mentioned earlier is really what the new 5th level of working hopes to alleviate.  

Through the use of AI Generative Systems.  Implementing automatic intelligent Generative Art Strategies.

The power of AI Generative Systems is that they can intelligently create presets for you on the fly. A literally infinite variety of them. And then they can intelligently work with multiple kinds of presets, working with them together in sequence to build more elaborate effects.  More elaborate generative strategies.

So it's like scripting, except Studio Artist builds the script for you, automatically, on the fly, and it can always be different if you want that, even as it follows a generative strategy that is constrained in some high level conceptual way.

So AI Generative Systems are the 5th level of working. Building on top of everything below it. Built on 20 years of development. So many old effects to rediscover. So many new effects to learn about and explore. Letting the program build artistic strategies and presets for you for you.

Or letting you define artistic strategies for it to explore for you automatically. Like that crew of really hard working people famous artists use to crank out their work for them.

Gallery Show has expanded tremendously in Studio Artist V5.5.  Originally conceived as a way for artists to create free-running dynamic visual art shows using Studio Artist in an actual art gallery (it was very lightly used for this task), people quickly glommed onto it as a way to visualize what the factory preset collection did, or to batch process images with custom PASeqs, or to riff random mutations of pre-existing presets.

In Studio Artist V5.5 you can use Gallery Show to build custom Generative Art Strategies.  Generative Art Strategies that can be fairly simple, or extremely elaborate.  The new Generative Paint options in Gallery Show basically allow you to control Studio Artist as it dynamically creates an infinite variety of custom automatically edited paint presets on the fly.  Exploring areas of the overall editable parameter space you might never get to through the use of old school manual preset editing.

We've also tried to bring many of the intelligent Gallery Show generative features up to the surface, providing ways to access them at the high level working interface of Studio Artist.  Through new smart Quick Edit command options for the paint synthesizer and the vectorizer.  With more to come in the future.

Old school preset editing involves going into the depths of the Editor for the specific operation mode (paint, vectorizer, ip op, etc), and manually editing individual parameter values by hand.  This extensive level of editing control over every effect available in Studio Artist has always been there, and is great from the standpoint of being able to tweak things to your hearts content. Dial in just the effect you want.

The other side of that coin is that you really need some level of conceptual understanding of what is going on under the hood to really be fluid at this low level of manual parameter editing for visual effects and digital paint.  And if you aren't carrying around a conceptual understanding of how the paint synthesizer works internally, how the vectorizer works internally, hand editing all of those parameters can be a very challenging left brain kind of task. A task that gets in the way of your right brain creative work.

At the same time, it has become quite apparent to us that increasing the level of internal visual attribute modulation inside of these areas of the program (ie expanding the level of editable parameters even more) leads to amazing visual output effects.  A whole new level of visual wow in the resulting art output.

So that is the dilemma.  Expanding the range of editable parameters increases the range of potential effects, and increases the resulting visual complexity of the effect output, pushing it to an amazing new level. But the program becomes more unwieldy to manually edit, even for experts like myself well versed in how it works internally.  

AI Generative Systems to the rescue.

Rather than having to manually adjust individual editable parameters, we've been adding all kinds of different intelligent Quick Edit commands. Smart edits that let you work at a higher conceptual level when editing an existing preset, or when creating new ones from scratch.

We've started this process in V5.5, but it's going to be an ongoing endeavor from now on.  More of them, and more places to use them.  You can run these smart Quick Edit commands manually.  But you can also use them to build automatic generative strategies that can run within Gallery Show in V5.5.

Let's say you want to use MSG Brush Load (did you even know it existed, or if you did how to manually program it in the paint synthesizer).  You can just dial up that generative paint option in Gallery Show and Studio Artist will then riff out an endless variety of new unique most likely never seen before paint presets on the fly that are all using MSG brush load internally.  

Studio Artist is automatically creating new presets for you on the fly.  But they aren't just total random mutations of the parameter space.  They are constrained in intelligent ways based on your personal specification.

Let's say you want to edit a paint preset to make it 'wet'.  That could take a number of very specific manual edits within different control panels inside of the paint synthesizer Editor if you wanted to do that manually. For just one particular approach to building a wet paint.

Now you can just dial up that generative paint preference in gallery show. Or you can run a specific smart Quick Edit command to do it for you at the top level working interface.

Applying specific effects (paint, ip ops, vectorizer, whatever) to specific areas of the canvas while not processing the rest of the canvas with that effect is an important component of building an overall final art image.  Certainly not essential, but extremely useful.  A great creative tool to take advantage of in your work.

You could always do this via manual selection and subsequent manual masking of an effect in Studio Artist.  But Studio Artist V5.5 can do it for you automatically now. Intelligently deriving an automatic selection mask on the fly.  You can dial this in with all kinds of different variations in Gallery Show, or you can run specific variations of it up at the high level working interface.

Is the 'intelligence' associated with this perfect? No.  Is it useful? Yes.  Will it get better as we move forward into the future?  Absolutely.

But what about movie support?  Is that back.  Yes it is.  You can now work with movie file io again (as opposed to being forced to use folders of frame images like you had to in the 64 bit build of V5 on mac).  You no longer need quicktime for windows if running on windows.   

Are all of the codec options you want available now?  Maybe not.  But we intend on expanding those as we move forward.  Yes we know you want ProRes.

We also want to add support for MNG in the future for movie brushes, since it provides a great open source container for lossless PNG images with embedded alpha channels.

Live video is also back in somewhat limited form. All of that will be turned on as we move forward, and we have some interesting ideas for expanding what people can do with it.  Multiple video sources being one of them.

What about other new stuff?  Well, there is quite a bit of it. Sprinkled all over the program.

There is a new positive/negative space visual attribute available throughout the program where ever visual attribute modulation is available.  It models the visual perception of positive and negative space in an image.  Since we now offer visual attribute modulation options for both the current source as well as the current loaded style, it's available for source and style.

We've built in this whole new conceptual notion of clipping an algorithm internally.   You can think of algorithm clipping as like working with selection masking, except the processing algorithm is masked internally as a part of its under lying internal structure.  As opposed to after the fact by masking what gets placed from the effect output into the working canvas.  Which is how canvas masking works.  

This is currently available in the paint synthesizer and the Vectorizer.   But you can expect to see that expand to other areas of the program in the future.

The Transition Context features have been greatly expanded (did you even know Transition Contexts existed).  More algorithm options, adjustments for them now, and the ability to route where the output of the Transition Context ends up.  You can route to the canvas, source, or style.

Transition Contexts automatically generate transition effects between different keyframes.  Each keyframe is associated with a specific image or movie file tied to that keyframe.  

Dual Mode Paint now lets you use the Vectorizer as a DualOp if you so desire.  We also introduced the concept of injecting time based modulation from Dual Mode Paint into an effect that overrides existing internal parameters.  This currently only works for Vectorizer Dual Ops, but is going to be expanded out to the other Dual Op choices in the future.

This override injection modulation feature is going to become very important as we develop the program into the future. Because it provides a way for other effects or signals to modulate an Op Mode effect without having to add endless internal parameters for those additional modulation options to a specific effect. 

We discussed Load Style in the initial discussion above.  Current support at V5.5 release is a loaded style image, but loaded style image folders are coming soon.

We also discussed Gallery Show in the initial discussion above. Many new Gallery Show features are available now.  New techniques, new paint transform and path path options associated with paint based techniques, a whole new generative paint preference tab with associated options, new generative options associated with vectorizer techniques, new source options for gallery show, new automatic intelligent masking options for gallery show, new built in generative options for the Start and End cycle processing, a timer to skip to the next cycle if using paint techniques that would otherwise run a long time, new automatic color palette generation options.

Many of the new gallery show generative options are now available in the high level working interface.  This includes the paint and vectorizer generative preference options which are currently available as QuickEdit commands in the Edit menu, automatic selection mask generation options available in the Canvas : Selection menu, paint draw and paint path generation options available in the Action : Art Strategy menu, generative dissipative image processing available in  Action : Art Strategy as well.

There is a new Gallery Show Toolbar you can use to control gallery show in the high level workspace interface.

There is also a new PowerTool Bar that currently supports some of the gallery show generative features when working in the high level workspace.  Our design specs have detailed descriptions of additional features that will be available in the PowerTool Bar in the future (like the ability to work with sets of PowerTool presets, and the ability to access QuickEdit commands directly in the PowerTool Bar).

There is a new M mutate button in the Preset Browser that lets you mutate new presets on the fly based on the currently loaded factory preset category.

We mentioned you can internally clip the vectorizer algorithm, which is currently available for region generation as well as region color modulation.  The old Draw control panel is split into 2 now, Draw Setup and Draw Apply.  There are a ton of new region modulation options in Draw Setup now.  There are new modulation options in the Draw Apply control panel for color gradients and cast shadows for vector rendering.

We mentioned you can internally clip the paint synthesizer algorithm. This is currently available in Paint Color Source as well as for brush Nib clip in Paint Fill Apply.  There are 2 new pen modes, freestyle multi-pen and freestyle region fill as brush.  You can embed vectorizer and ip op effects that generate vector output directly into a paint preset, and then use the resulting generated vectors for path sketching with paint or path start regionization.  You can choose whether the selection buffer is overwritten by individual regions when running path start regionization or not (it always did it before).  You can now chose whether the aspect ratio is maintained for resized image or image folder or movie brush folder paint nibs, or whether they resize to fit like they used to.  You can choose whether the visual attribute used for path angle modulation is based on the source or style image (more style modulation is coming in the paint synth in the near term future). New Brush Size modulation options.  Bezier paths can now store and restore brush size in addition to brush color.

We've got some experimental optical flow stuff going in Temporal Ip Ops that is going to be cleaned up in the short term.

There are various new IpOp parameters and features hiding inside of some of the individual IpOp effects.

Studio Artist V5.5 is based on a totally different internal framework (different than V5).  This framework is very current with recent OS changes, so things like cosmetic issues on mac Big Sur are gone.  It supports things like dark appearance display mode on mac.  This change may seem trivial to you, but believe me it was not trivial under the hood to implement.

There are various other new things i'm spacing on right now that i'll try to drop into this list as they come to mind later.

So, i hope this gives you some idea what the long awaited Studio Artist V5.5 release is all about.  And where we have pointed it for future Studio Artist development.  

We've also been working internally on V6 things as well, so that effort has been ongoing in the background and will now move to the forefront with the V5.5 release.

Moving V5 Movie Brushes to V5.5

 Porting old Studio Artist V5 to V5.5 on the mac has been quite an adventure.   For many reasons, but today we're going to focus one specific little area. 

All versions of Studio Artist prior to V5.5 were internally based on apple's quicktime api.  Which in their infinite wisdom they gutted with a knife and killed off when they decided to totally bail on 32 bit applications and 32 bit code apis for developers.

But why didn't they just port the quicktime api to 64 bit people oftentimes ask me. I haven't worked at apple for over 20 years, so we can only speculate. I have however seen the actual quicktime code back in the old days, and it was all based on old style GWorlds (GWorlds are messy). So yeah, i can understand why they wanted to get rid of that. 

So why didn't they rewrite the inner workings, and then attach that to a new set of higher level api calls that closely mirrored the structure of the old ones i then get asked? To make porting people's old code that used the old quicktime api calls easy when they wanted to port it to 64 bit.  Yeah, real good question.

Instead, they told everyone to switch to a separate api called QtKit that was 64 bit.  Of course QtKit seemed pretty half-baked, missing important features, and was then unceremoniously dropped later on when they killed off the quicktime api. So good thing we didn't waste our time doing that.

Instead they switched over to something totally new called AVFrameworks. With no C++ api calls. Instead, every C++ developer could go try to implement their own C++ api headers that tie into the objective c code AVFrameworks lets you tap into. Because why would apple tell one apple engineer to do this work for all their developers when they can instead just make every 3rd party developer do it themselves.  This is a repeating pattern occuring over and over again int he apple - apple developer relationship, or i wouldn't be so bent out of shape about it.

But of course apple actively tries to discourage C++ development on their platforms these days.  In subtle and not so subtle ways. Even though none of their historical long time developers used it when they started that push. They want you to use either objective c and Cocoa, or Swift these days. Probably because both options essentially lock you into the apple platform with no real way to escape.

So we see what is a re-occuring theme here, screw you existing developers, apple knows best.  We also see another re-occuring theme here, apple loves to kill off the past. As in all of your previous work.  Apple wants to torpedo it, make it non-functional, pretend it never even existed.  And they do that over and over and over again.

Gee, that sucks for developers you may be thinking, but i'm a user, surely apple loves their users.

Does apple really love their digital media content developers and digital artist users? Apparently not. because when they killed off the quicktime api, they also killed off most of your old movie brushes (and all of your other old quicktime based media content).

Now how they did this was one giant fuck you to all of their old time digital media content users that used quicktime to archive their work.  Because many of those people used quicktime codecs that are now no longer supported on apple's platform.

A classic example of this is the old quicktime animation codec. It allowed for lossless compression at the highest quality settings along with support for an additional embedded alpha channel.  All of our old factory movie brushes used this codec for exactly those reasons.

And if you made movie brushes you used in Studio Artist, you probably did that as well.

You probably also archived final work, both from Studio Artist as well as other software products in various quicktime codec formats that are not longer supported on the mac platform. No longer supported beyond Mojave.  These files are files of nothing on Catalina, Big Sur, Beautiful Fremont (our pick for the next mac os name).

Now i can totally understand why they might not want the old animation codec code to dirty their spanky new GPU engine for live streaming video. Totally get it. But i find it odd that the company with one of the world's largest capitalizations, the company that had no trouble trotting out Rosetta 2 emulation to run intel code in emulation on ARM Risc chips, i find it odd that adding import for a run length encoder movie codec was too technically advanced to implement. 

Or perhaps they just didn't care.

That last answer is the correct one. They didn't care about the over 20 years of quicktime encoded source mater, final projects, movie brushes, etc that their long time users had archived. It just never even existed.

So, what is the poor digital artist that used macs forever to do with all of that old quicktime api encoded material. Obviously you need to keep a mac running Mojave around forever (good luck on that one) to even read it.  Because if you want to convert it into something that is readable on Catalina or later, you are going to need it.

Apple's deceptively named Quicktime Player app (deceptive because it does not use the quicktime api, nor read most of the previously acceptable quicktime file api codec formats) will open those old no longer supported quicktime codec encoded files on Mojave (not on Catalina, Big Sur, Beautiful Fremont, etc).  It brings up a dialog saying it is 'converting' them.  And then you can save those 'converted' files.

It saves them as H264 mov files (you don't have a choice in the matter). If they had an alpha channel, that is magically missing now. And if the files were lossless before, they sure as shit aren't now (they look pretty horrendous).  You might have better luck with the old Quicktime Player 7 Pro app.  Which was actually a really great as a swiss army knife utility for working with quicktime movie files, and allowed for a full range of export options (unlike the brain dead newer Quicktime Player app they ship these days).

So, if you have old movie brush files you used in Studio Artist that didn't come with our factory set of presets, you are going to have to convert them.  We converted the old factory ones for you, and they are in the Brush folder of Studio Artist V5.5.

Now if you want to use those converted movie brushes on windows, you are going to have to do a second file conversion (thanks apple?). Because the H264 mov files that apple exports on Mojave are not going to work both in the Windows Media Player and in Studio Artist V5.5 on windows on most (bizarrely not all) windows computers.

Fortunately, you have Studio Artist V5.5 to the rescue.  Because if you just pass them through Studio Artist V5.5 outputting to H264, then they will work on all windows 64 bit win 10 computers. In the Windows Media Player, as previews in the windows finder, and in Studio Artist V5.5 on windows.

But now we get to another apple gotcha.  Because many of you might already have a Studio Artist 5 folder in your Applications folder on Catalina or Big Sur (Studio Artist V5 64 bit runs on both).  And any old presets that reference those old factory movie brushes reference them with a pathname like this

"/Applications/Studio Artist™ 5/Brush/Movie Brushes/WackBrush1"

Now Studio Artist is smart enough to search in your local Brush folder located next to your running Studio Artist application if it can't find the pathname file reference for the movie brush.  

But of course if you have both a Studio Artist 5 folder and a Studio Artist 5.5 folder in Applications, then Studio Artist V5.5 is going to find the old totally valid movie brush pathname file reference, and try read in the old movie brush file that can no longer be read on Catalina or Big Sur.  And you wonder why things aren't working until you figure this out.  It's very annoying.

If you add an extra letter to the name of the old Studio Artist 5 folder name, that will prevent this from happening.

Studio Artist is smart about searching for any image or movie brush files inside of the local Brush folder if the original file reference is no longer valid.

But of course we are not done with apple gotchas.

Because for some weird reason, when Studio Artist V5.5 tries to open totally valid movie files that are stored inside of our Brush folder inside of the Studio Artist folder (which is inside of the Applications folder), we can't open them on Big Sur.  

But if you either move the Studio Artist folder somewhere else (like in your Documents folder), then the movie files open fine. Or if you keep the Studio Artist folder inside of Applications, and instead reference a movie brush somewhere else in your filesystem, they open fine.  This is true for both movie brushes and movies you want to open as the source.

Now this is a conundrum,  since we store Studio Artist assets like Brush assets inside of the Studio Artist folder.  Rather than installing them in weird hidden locations (like /users/yourname/Library/ , which is a hidden folder for normal users that can't even be seen in the Finder unless you use special tricks).

I think we're going to add a preference option to retarget your brush folder to ultimately resolve this.  But it's still not ideal, because i would really prefer to keep the drag and drop installer rather than move to some more elaborate installer that dumps stuff at various places on your hard disk.

My take on all of this is that movies used to be awesome on the mac, and now the whole movie thing is one giant mess on the mac. Things work differently from os version to os version, essential utility tools that were freely available on the platform are just abandoned, all of your old work is rendered useless.

And the crippling of essential file system file access features is also just one giant mess. I view it through the lens of apple slowly turning the mac computer into just another locked down information appliance.  A computer is an engine for the mind. An information appliance is a purposely crippled widget, primarily designed to suck money out of you while also providing some additional value to make the spending of money on it seem useful.  Useful for some tasks (web browsing, watching a video, reading a kindle book, etc), but generally restricted and limiting in so many ways.

The new 'security' features are ridiculous. A user specifically navigates apple's own standard file dialog to access a file in some specific part of the file system, and then a parental warning message comes up asking you if you really want to access a file in Documents? The user just specified that by working with apple's standard file dialog to get there? If they didn't want to access a file there, they would not have used said standard file dialog to navigate there in the first place.  The endless nagging is tiring pretty quickly.

And of course the nagging doesn't stop there. Because every day your essential Mojave system you absolutely need to read all those old file formats no longer supported will try to trick you into updating to a newer os version that will kill off the ability to read those old files. Because apple doesn't care about you or your precious old work.  They would much rather prefer that you buy all new applications from their app store so they get a 30% cut of the revenue, and that you do that every new os release. Preferably using their apple credit card, so they get an additional cut.

It's a sad statement on what used to be a great platform (indeed used to be the best) for digital artists and digital media developers, both users and developers of those tools. I can no longer say that is true.

Why would anyone ever depend on an apple proprietary file format every again. Or an apple proprietary code api for that matter.

So there is a view into the giant mess that apple has rendered unto digital artists on their platform.  Meanwhile windows keeps looking better and better, which is something i thought i would never say.

It's really a shame, because they did a fabulous job with the new M1 ARM Risc chips.  But their software, ouch.  And their attitude, double ouch.

I think we're going to be moving to support MNG file format for movie brushes in the future as an alternative to the existing support we provide.  No dependence on apple computer, so they can't muck it up somehow next year when the new Beautiful Fremont mac os hits the ground running.

We support image folder brushes, so there is a totally acceptable alternative way to get there (lossless compression and alpha). But there are advantages to having one file rather than a whole folder sometimes for things like a movie brush.  MNG is an open source container file format for PNG files, so it's a great solution for lossless movies that also fully supports alpha.

Dancing Diodes Music Video

 Studio Artist user Damping Factor has released a new music video for their audio track titled 'Dancing Diodes'.  Most of the visual effects in the music video were generated using Studio Artist (pre V5.5 version).  Let's check it out below.

Cool music and video, so thanks for sharing.

In watching this, i thought it might be cool to revisit some of the old colorized edge detection filtering effects and redo them as vector.  Effects that have been around for awhile like Color Edge or Edge ip op (or even Gradient with it's parameters dialed in for this kind of look) now have vector output options in addition to the old raster ones.

But you can also now embed any vector ip op effects inside of the paint synthesizer. So the vector output of the ip op effect is available to be used as the internal bezier paths for the painting.  The IpOp effect intelligently derives a set of vector paths from the source image (or whatever else you push though it (canvas, style, selection, etc).

But then you could also turn those IpOp embedded paint synthesizer actions into paint actions that contain an embedded bezier path frame.  

In the former case, the bezier paths are derived live from the embedded IpOp when you playback the paint action step.  In the later, the bezier paths are directly embedded inside of the paint action step.

The former case will paint differently if the source image is changed.  The second later option will not change how they paint if the source image changes.  Although you could override the passed through coloring of the embedded bezier paths from a different source or style, but the path that paints on would always be what was embedded in the paint action step.

Now the cool thing about paint action steps with embedded bezier paths is that they can keyframe over a series of successive frame times from the start keyframe to the end keyframe of the local path interpolation.

And Studio Artist V5.5 has different options to synch up the # of bezier paths and the individual path synch up ordering for the path interpolation as you mover from keyframe to keyframe.

So i think that would be a fun alternative to the more old school raster video edging styles.

We'll check out the specifics of how to do all of that above in a future tutorial post here.

Studio Artist V5.5 final build hit

 So Studio Artist V5.5 has reached that nebulous stage in the development of a software product where the code is frozen, it's the final build. No code changes for the Studio Artist V5.5 release.

So just packaging and distribution issues to finish up.  Bear with us during that hopefully brief period.

Is it finished?

No software product is ever finished. 

Can you get things done with it?

Absolutely.  I have 1000s of amazing visual images generated using just the generative strategy features of gallery show.  Stuff i generated during what were essentially automated stress test runs (as opposed to more focused artistic driven sessions).  With not very much thought or effort put into putting together accompanying folders of presets specifically designed to enhance a particular generative art strategy.  

Manually manipulating the program is of course a whole other universe filled with endless exotic destinations to explore.   Lot's of new features hidden away in the depths of the various Editor control panels.

Is it finished?

Absolutely not.  But we have reached the point where we need to get the 5.5.0 version defined, and thereby exit code lockdown, so that we can bang out a few fixes for issues that came up during our stress testing regime.  

So we anticipate a fairly quick turn around on an incremental 5.5.1 update, that will be released when it's finished.

When will that be released?

Ha, ha.  Like i said, we anticipate a quick turn around, but that's the best i can tell you at this point in time.

We want to do a clean sweep of the issues we're aware of.  We'll try to make you aware of them as well, to aid you in the use of the 5.5.0 build until the 5.5.1 build is available.

Some of these issues are really apple curve balls (they do like to throw them at their customers and developers).  And in some sense, i really feel like they are introduced because apple is trying to make the mac act like an information appliance (iphone, ipad) rather than an actual computer.  

Issues like arbitrarily restricting access to the file system.  Like it's some dirty place you aren't supposed to go.  As opposed to being a great way to hierarchically organize information.  Where both the program and the user can interact with that structure and it's contents.  One would hope easily, without undo restriction.

Like movie brush paint or dual paint or PASeq preset that reference a movie file inside of the Brush folder we ship inside of the StudioArtist folder in Applications as not being openable to the StudioArtist application, even though the exact same movie files work fine in the preset file if you put them some other place in the file system and then edit the preset to reference the new location. On some of the mac systems we support, not all (of course).

For some strange (aesthetic?) reason (why exactly?), apple decided to not put the standard file dialog titles up in the title area of the standard file dialogs.  Just for external application developers using the pre-defined interface to those standard file dialogs,  but apparently not for internally developed applications.  Reminds me of how they put the plug to charge their trackpad mouse on the bottom so no one could use it while it was charging (but it sure looked nice that way i guess?).

We can either move to non-standard file dialogs (which would solve the issue today but at the cost of a change to the normal visual appearance), or wait and hope the issue goes away when we move to our next internal framework.

Of course non of these apple curve ball issues are a problem for windows users of Studio Artist V5.5 on windows.

Other outstanding issues are associated with features that need to be finished (or turned back on).

Live Video background textures are currently turned off (did you even know they existed?). In conjunction with this, you can grab a live video source manually, but the live preview then capture option is turned off as well.

Like Temporal Generator (TG) functionality in MSG Brush Load and MSG Live Source Brush.  We turned it off to aid us in tracking down a wild MSG bug in a pretty obscure place that was probably never visited in the age of manual parameter editing of Studio Artist effects.  There are also a few lingering turned off msg source color palette features in those control panels.

But the generative strategy algorithms in V5.5 automatically edit the application's internal editable effect parameters in ways you could only dream about doing manually in previous versions of Studio Artist.  So you do end up visiting those back water corners of the editable parameter space (and believe me there are some seriously wild ones).

So super obscure bugs no one ever ran into suddenly pop up into view.  And we've been tracking all of these down.  As they come to light.  I think there are a few still waiting to be found, so we're continuing our automated hunt.

The Load Style features need to be finished up.  The ones we envisioned for V5.5.  You can extrapolate away about where that particular feature is headed in the future.  It's an amazing journey we are all going to take over the next year.

So there are a few specific things we want to finish up or turn on associated with Load Style.  Like folders of images as an alternative to a single image for your Style.

The Style Palettize option in the sub menu of options when in the Palette tab of the source area is not doing what you would expect it to do. So we'll fix that. 

Here's how to get there quickly manually.  You can use Edit : Palette " Generate from : Style Image menu command to generate the source color palette from the currently loaded style image.  Then you use the Palettize sub menu option in Source Area.

Style texture visual attribute modulation in the paint synthesizer is defaulting to style luminance right now.  That will be turned on.  We may also open up some more style visual attribute modulation features throughout the program.  Look for the hidden ones already existing in Ip Ops.

The empty PT tab option in the Preset Browser is basically a place marker for currently turned off features of the new PowerTool Bar.  Turned off features that have to do with letting you access presets inside of the PowerTool Bar.  

The optical flow features need some work.  We definitely want to make this area really shine.

The PASeq preset file format version id needs to be bumped up a notch to catch some late arriving features in Transition Contexts.

There are some outstanding PASeq playback issues for the 2 new pen modes in the paint synthesizer.

Things like the paint Synthesizer and Vectorizer Smart Edit commands need to be expanded.  You can never have enough, but algorithmically manipulating the style modulation is going to be really important.

As far as lingering bugs, i think there is still something hiding inside of PASeq playback during PASeq based gallery show runs using the Factory PASeq presets.  There is some concern that there may be old presets inside of there generated during development times when the version ids were changing quickly, (and perhaps got our of synch?), thereby causing havoc when they are read in.  We're actively working through it to make those go away.

So theres a quick overview of the kind of stuff we want to shine up for the upcoming V5.5.1 update release.