MACHINIMA MANIFESTO MARX CATTENEO DECEMBER 2013
"cinematic, actor based, machinima in Second Life anno 2013"
Dear reader,
I am Marc Cuppens aka Marx Catteneo (SL name) I work as a photographer and short film maker in real life and in Second Life (SL). I have made proposals to Linden Lab (LL) for improvements of the SL viewer in the field of machinima (film shot in real-time in game engines). I have worked jointly with developers of third party viewers.
I worked first with KirstenLee Cinquetti on the discontinued Kirsten's viewer, the first viewer to have real-time shadows (the "film menu" was my creation). When Kirsten's stopped being developed, I made many improvements to my menu and i added a window (Machamera).
Around the end of Kirsten's viewer, Willam Weaver also was working on what later became Phototools in FireStorm. William and I discussed it and had a different view on usability of the User Interface (UI) so there was room for co-existence. Making these kinds of additions to viewers means a few months of testing every existing debug setting (some of which even the Lindens I asked about these are clueless what the settings should do or if they still work) as well as trial and error to implement everything into a useful menu or window. I have no real programming skills, so I couldn't make everything work as some desired features would need "hard-coding".
Machamera became an add-on I could set to work in every modern viewer by replacing and or editing a few XML files in the program folder. After testing it in the existing viewers back then, I chose Dolphin because of its stability, smooth operation for camera movements, update frequency and closeness to the official viewer's UI. After i spoke to Lance Corrimal he implemented Machamera, with a few UI changes to his taste, in his official releases. Dolphin is temporarily no longer being updated so nowadays I use Exodus as my main viewer because of its closeness to the LL UI, its very interesting extra features for visuals and its rendering and rezzing speed. It works very well with MaChamera added.
Year 2013.. at the start of the year I am working (in SL) as Director of Photography on Poppy Dreams (directed by Kit "Gish" Guardian) SSA/SSB (Server Side Appearance/Baking a major change on the server side) is approaching and SL becomes very unstable. The first problems we noticed were jewelry or shoes and even walls not showing after rezzing or after a camera movement, this has slowed us down a lot because we had to check actors and the set before every shot and during camera movements.
Apart from this very nasty problem, working for weeks in a row on this project, showed me the wide range of problems SL throws at us when you want to do serious, actor based filming with top quality graphics settings and realistic lighting. Later that year i was granted a full sim for my own machinima project Nin9, a web-series 9 episodes filmed with a cinematic and experimental style. To my disappointment, as well as that of all people who cooperated on the project , I have to conclude that I can not finish my Nin9 project in SL. In the text below I will explain which problems make it impossible for me to do the project in SL. I sincerely hope readers from the Lab will take a look at this Manifesto and hopefully plan to solve some of the problems because SL is so close to being a serious tool/platform for very stylish machinima.
Most problems described are avatar related and I have to be positive about the potential SL has for non actor based films such as art installation registration or a tour of a building. Many recent improvements add to a far "richer environment", I do praise LL for many changes they implemented, despite massive user base resistance against these forms of progression.
DEFERRED RENDERING A PARTLY FINISHED GIMMICK OR SHOULD IT BE THE MEASURING STANDARD?
For those of you who don't have a great graphics card (SL demands ridiculously high hardware specs compared to modern games) Deferred Rendering also named Advanced Lighting Model (ALM), is a "new" way of rendering the environment, it means we can have a theoretically unlimited number of light sources (old had maximum 6) also the way the light sources spread their light is much improved and real-time shadows became possible as well, usually anti alias is also impossible if ALM is not on. Later Depth of Field (blurred backgrounds) was added to ALM and this year we saw the new material system which is a huge improvement.
If you want to create cinematic footage you have to either add all this in postproduction or just turn on ALM during the shoots.
Turning on ALM gives us a much better image but also reveals some problems, for instance prim parts on pants are lit differently than the avatar mesh is, so those will stand out, this goes for many of the old school solutions to the limitation problems of SL. Feet/shoes always have been a problem in SL, the old school solution was to add huge invisiprims to hide the ugly (too few rectangles mesh) avatar feet, this was showing when the floor or textures behind had transparency, you could see the "invisible" blocks around avatar feet. In ALM these invisiprims are basically ALWAYS standing out, another problem that really stands out is facelights, some people wear up to 10 of them (while only 6 actually worked in the old lighting system) they pretty much look like a nuclear explosion to those who have ALM on.
The huge difference in how things look for ALM and non ALM users is a problem because the non ALMers have no clue they look so bad on camera, when i'm working with actors i usually have to spend an hour fixing all those problems before we can get to work.
THERE IS NO MANUAL FOCUS
Depth of Field (DoF) has been one of the greatest improvements of the viewers in my opinion, finally SL-films could have that cinematic look I always wanted, but... like many real life consumer video-cameras LL didn't give an option for manual focus. In July 2011 I created a feature request which in my opinion was too easily waved off to third party developers by LL. https://jira.secondlife.com/browse/VWR-26201, PLEASE VOTE AND FOLLOW IF YOU AGREE WITH THIS
Dear LL, when you implement a feature such as DoF please finish the implementation, in the Jira I give the complete solution and free hotkeys. What you have given us since its introduction is a cheap handycam while with a few simple changes it could be a professional broadcast quality videocamera.
ALT ZOOM FOCUSSES ON THE COLLISSION BONES/BOUNDING BOX NOT THE AVATAR TRIANGLES
Actually sort of part of the first manual focus issue but for moving avatar filming worth spending a separate paragraph on.
Try moving the camera really close to an avatar's face instead of the amazing photographic effect of shallow DoF and a controllable sharp region what you get from a distance under 1 meter or so is a blurred face this is because the focus is not on the actual avatar skin (mesh triangles) but it's on a bounding box. This makes it impossible to do an extreme close up with alt zoom.
Why not use space navigator for that.. well that works better because the focus will actually be on the real object "under the cursor" but when an AO or avatar movement moves the avatar you can't keep track of it in such a close up.
Altzoom can keep track of an avatar, you can lock the cam to it though this actually locks it to the box again so when the avatar starts moving or changes pose the camera will not follow correctly.
While these problems might seem not too big for photographers they are actually close to showstoppers for machinimators, especially combined with the lack of a manual or lockable focus.
When I want to show an avatar having a drink while he should better not do that, it would be a great shot to track the hand with the glass moving towards the mouth , and with DoF there will be extra emphasis on this action, I won't bother trying to film such a shot in SL. I can't lock on the glass (probably because it's an attachment) and locking on the avatar will not follow the hand movement, the animation is way too fast to follow with space navigator so it will only be possible to film this with my own avatar (see the paragraph slowing down animations).
HAIR AND OTHER TRANSPARENCY PROBLEMS IN ALM
Since the introduction of mesh hair we have seen very nice hairdos but also a huge problem, when the creator uses transparency on a 3d shape like a cylinder the inside of the cylinder will become totally transparent even for DoF, so when i make a closeup of a face with such hair there will be a halo of sharpness making 95% of the mesh hair useless.
Shoes and feet in SL, the biggest frustration for a decade, even though we have alpha layers now many people don't ever use ALM as i always use it i see many people with the ugliest feet and they are clueless it's so bad: what you see in ALM is the heels or toes poking through and the invisiprim has the same problem as the transparent hair making half of the avatars -off the shelf- useless for cinematic shots.
Glass and other semitransparent objects present another related problem, when there's a glass in front in a scene and the focus is on the glass the scenery or an object/avatar behind the glass will be in focus even though it is at a distance that should make it blurred.
AVATAR MOVEMENT TOO FAST AND JERKY
When an avatar moves from point A to B we have walk and run, there should at least be added a stroll, at about half the covered distance from what walk does. Ideally it would have a setting that allows user defined speeds.
Walk is still very hard to track with the space navigator and as seen below tracking with a locked cam is not good enough anymore so it's impossible to film a walking closeup. Apart from the speed issues there also is the lag problem (if it is lag causing this) it's also known as rubber banding: for the actor the movement can seem smooth but there can be some jumps back and forth for the camera, so often multiple takes of a shot are needed to pick the least problematic one in editing. Huge time consumer for machinimators!
AVATAR MOVEMENT WITH LOCKED ALTZOOM CAM NO LONGER SMOOTH
In 2009 I could make a reasonably smooth tracking shot of an avatar walking or flying, it could take some tweaking the "position smoothing value" but since 2012 (guesstimate) this is no longer possible to set up properly in any viewer i have tested. When the cam is close to the avatar (from about "a total" in movie-lingo and closer, which is not very close at all) there will always be some twitches and jerks for the camera.
-edit jan 2 2014
NUMERIC INPUT FOR AVATAR ANIMATIONS SLOW DOWN / SPEED UP
Untill miguael liamano (Voxhyr) pointed me to the fact that the V3 viewers have a menu item for speeding up and slowing down anims for all avatars i thought it didn't exist. Now i do wonder why I didn't "get this memo" or why LL didn't brag about it in machinima groups as this is absolutely big news for machinimators! We can do "The Matrix" like slowed time shots now!
I really love this and the fact it now exists takes out 2 of the items from my list however this wouldn't be a manifesto if i didn't have at least a tiny comment on the implementation. I would have loved it even more if it had any kind of feedback on the current speed or simply a numeric input or slider.
For those who also wonder where this gem would be: Develop-Avatar-Animation Speed- 10% faster / 10% slower / Reset all.
The "slow motion animations" in this menu block still only works for the own avatar
-edit jan 2 2014
ANIMATION LENGTH LIMIT TOO SHORT
The above mentioned added speed control makes this no longer a valid point of critique.
LIPSYNC NEVER UPDATED TO A DECENTLY WORKING SYSTEM
How many years do we have the Lipsync babble loops system? No matter what kind of sound the microphone picks up the avatar lips will start moving randomly. The october 2009 article tells us to stay tuned for improvements, it's about time to have a few of those and while you guys are at it, why not hook up to a more stable voice service as well, this one fails too often.
NO FINGER AND FACIAL ANIMATIONS BESIDES THE DOZEN OR SO AVAILABLE
In December i attended a meeting for mesh creators held by Nyx Linden. As I was already plotting this text I was very curious about LLs future plans, (more about mesh deformer/fitted mesh later) During the meeting someone asked about finger bones (the possibility to animate the separate parts of the hands) if this was made available machinimators could have avatars make very realistic gestures. To my disappointment Oz Linden said finger and face bones will not happen anytime soon and are not in any future road map.
This also means that we are and will remain limited for faces to the dozen or so emotions that are provided now, they are not very subtle and for machinima to work with such a limited set makes many films look similar when trying to express any emotions at all.
MESH IMPLEMENTATION
Another example of a poorly completed project, when I look around at an average party I see 1/3rd of the avatars with holes in their bodies where there should be skin, badly fitting mesh clothes and when they do wear a full mesh avatar with tightly fitting clothes they have frozen faces and seem to be cloned unless they have spent a fortune on add-ons which is still limited to the imagination/time and technical possibilities of that particular creator.
The way mesh is implemented is damaging the unique selling point SL has, compared to other virtual worlds, people always had a very large degree of freedom on personalization of their avatars, giving us mesh without all body shape sliders working made us much less creative individuals.
NO MESH DEFORMER
I do know LL is currenlty working on "Fitted Mesh" and will soon have a Release Client with that feature enabled, from talks I had with creators I understand LL didn't exactly chose the best solution but, I'll give them the benefit of the doubt on this one. I am happy at least one of the major problems will get some sort of solution and I am eager to start checking it out, hopefully this implementation will see good results and LL uses the user feedback to improve he inevitable initial flaws.
RIGGING TO HANDS/ARMS NOT GOOD ENOUGH
When I had to rig the suit for my lead actor myself (try buying a fat man's suit in SL) I noticed some serious flaws in the rigging, at first I thought this was my fault so i kept tweaking it for a long time, many rigging problems got better but one problem seemed unsolvable: the wrist.
Weighting around the wrist is impossible to do perfect whatever I tried there was always parts of either an inner sleeve or flesh of the avatar poking through layers of cloth.
The commercial creators I showed this to all mentioned I did a good job so apparently this is an accepted flaw, but again for machinima close-ups it looks pretty bad.
PANT FLARES RENDERED AS TIGHT AGAIN (YEARS OLD BUG COMES BACK)
https://jira.secondlife.com/browse/SUN-121?
September 2013.. when I was about to start the recordings for the Nin9 project, to my big surprise some of my actors had tights while they should have had at least some pant flare. In 2011 I had to postpone the shoots for my short "Staying Alive" http://www.youtube.com/watch?v=yUwK_8-_sXg for many months because that was one of the first ALM shot machinimas in SL ever and I could not use prim flares cause they would stand out, back then there was a long Jira about this and many creators complained as well, it was solved by KirstenLee in his viewer and LL followed with a solution after that.
I was shocked to see this bug had returned again, as we now have learned this was probably caused by the transition to SSA/SSB but as we are in december 2013 now I still see the bug in various viewers on different computers. Hopefully this will be solved after the SSA/SSB bug fixes I heard being announced at the December mesh dev meeting.
ONLY 2 PROJECTORS CASTING SHADOWS
ALM now allows us to have shadows from the sun/moon and 2 projectors closest to the camera, though it works pretty good this is not enough for some situations in which a visible switch can be seen while the camera moves closer to other light sources. a shadow map as it is used in some engines could be a solution or a user settable max number of shadow-casters tucked away in debug settings would do for power users.
THERE IS NO DERENDER IN THE OFFICIAL VIEWER
I would not have had to go through all these viewers tests (various TPVs) if LL had made derender available in their own official viewer, the possibility to make avatars or object disappear temporarily is an absolute necessity for machinima. During my tests I often found that the official viewer is one of the fastest rendering ones and obviously it is the one that has the current UI, not loved by too many people but I prefer it over many TPV attempts to go back a few years in time to an (in my humble opinion) old fashioned look and feel. This is not a major issue for me as machamera will work in every viewer that has the xml structure and I will find my settings anyway though it can take some time and frustration sometimes to find out where a TPV has hidden certain functions.
VIEWERS HAVE NO MULTIPLE MONITOR SUPPORT
It has been on top of the list for machinimators, scripters and basically everybody for many years now. There is no way to have your HUDs or parts of the actual UI to be on a second monitor so they are always cluttering the screen and blocking the view. For machinima this is really bad because we have to remember to turn off the UI and HUDS and check it before every shot, it won't be the first or last time an otherwise perfect shot is ruined because there is something showing that's not supposed to be part of the footage.
LARGE SIZE MESH UPLOADS AND LAND IMPACT TOO EXPENSIVE
As opposed to sculpts mesh can get very expensive when resizing, as an example i had a nice shape that was 150 Land Impact (LI) in miniature version when blowing it up to 64 meters it became 1500 meaning a full sim can only have 10 of those to fill the entire prim space. There could be workarounds like cutting it up in smaller pieces (assuming you can make your own meshes at all) but this would mean so much extra work and time that LI is a very bad factor towards productivity.
The fact we pay twice for all objects can raise an eyebrow or 2 when you give that some more thoughts, we pay to upload (for a highly detailed mesh for machinima this can get pretty expensive) and if it's not an attachment we will have to pay for it to actually be present inworld.
IS A FULL SIM LARGE ENOUGH FOR MACHINIMA AND WORTH THE 300 US$ EX VAT PER MONTH?
Suppose i want to build a somewhat realistic city not just a single block but a small variety in areas, does that fit in a 256x256 meter sim costing 300 us$ a month? The answer is no or barely.. ALM doesn't allow us to use backdrops without it looking silly, so you can never have a street view longer than 256 meters or make a city looking like it has thousands of residents on an island of 256x256 meter, meaning for a convincing city one would need at least a double sim, if you build with very few details a double sim with normal sim prim amount could work but for highly detailed you will need the 15k prims on your normal sim surface meaning this city would cost 600 us$ a month. All those numbers do not include SL's setup fee which is very high as well.
A monthly fee of 300 us$ ex VAT would take us into the range of 3d rendering software like Maya, but SL doesn't even come close to the functionality such software does deliver to it's customers. Another comparison: Unity Pro has a monthly payment program that costs 75 us$ and this includes free updates or a one time cost of 1500 us$.
NO WAY TO MAKE A DECENT BACKUP IN A 300 US$/MONTH SYSTEM
Imagine building on a full sim with a small team, the objects placed are of various owners and of various permissions, when the time comes to end this sim to either move it to another one or pack it up for later use, it becomes painstakingly obvious this simply is impossible in SL.
Of course I do know there are perfectly good reasons why there is no way to make a backup of your own sim (permissions), but the bottom line is that it is totally unacceptable especially in a system with such a monthly cost.
SOLUTIONS:
- add derender to the official viewer
- fix the focus problems
- give us timing options on movements
- solve the transparency bugs/ make it look the same in normal and ALM
- make a better lipsync
- make a true avatar 2.0 and give the possibility to replace the avatar instead of adding mesh layers
- revive the puppeteer project and implement in the official viewer
- give an interface for custom alpha layer adjustments directly in the viewer
- add more shadow-casting projectors (user settable number)
- add multiple monitor support
- add some way to make a backup of a teamwork build
- make a cluster of sims available as machinima sandbox (extremely long return time) building for group members only (joining the group only after recommendation by members and review of quality level)
OUTSIDE THE BOX SOLUTIONS:
- Open the SL world for other graphics engines (opening for SFM "Source Filmmaker" will solve every problem mentioned in this entire manifesto) Logins with Steam are already possible so a first approach has already been made, this would turn SL into machinimators heaven!
- make a wormhole to other worlds (especially OpenSim/OSGrid)
CONCLUSION:
SL is in an eternal Beta phase, most games make sure machinima is well supported but for Linden Lab machinima seems not to be a point of interest, though they claim to encourage it. Ofcourse I wouldn't mind at all if my conclusion was to be proved wrong in the near future and I would be happy to further specify and elaborate on the above mentioned issues to any of the Linden staff or third party developers who feel they could work on those issues.
For those who managed to read it all I thank you very much for your efforts and wish you a great 2014
Cheers,
Marx/Marc