Commercial developers are now releasing music creation apps for mobile game systems, in the form of the KORG DS-10 for Nintendo DS and Rockstar’s Beaterator for PSP. But some of the best ideas still come from the homebrew community.

What’s most impressive about ToneSynthDS is not so much what it does as its interface, fitting all its functionality into the DS’ two compact screens. Its minimal interface finds an elegant arrangement of everything you most urgently need, with a sequencer screen on one DS screen and basic virtual analog synth parameters on the other. A 4 x 4 matrix next to the main sequencer grid lets you switch between patterns, in a step sequencer reminiscent of the monome and Tenori-On. There isn’t a whole lot of depth to event editing in this early version, but it could be a lovely way to sketch melodic patterns. (And some of those limitations come from the DS itself. Note, though, that this app gets a full 16 real-time channels on the original DS hardware to the Korg DS-10’s paltry two.)

Developer Fanta/Hotelsinus Sound Design has been posting mock-ups, demos, and now builds as he goes. That means that he gets feedback from an audience of readers and incorporates those as he develops the app – another key difference between the DIY/homebrew scene and conventional commercial development.

More good news: this DS app should also run as a PC VST in a forthcoming version, opening up the fun to folks using netbooks and laptops instead of the DS and creating a nice mobile-to-computer workflow.

http://ndscomposer.blogspot.com/

In related Nintendo DS news: If you’re thinking about getting the new DS-10 Plus Limited Edition of the KORG DS-10, you’ll need to get it for the region coding of your DS. (In other words, you probably won’t want to import it.) The “Dual Mode” functions are region-locked, so North American and European users can’t use the Japanese DS-10. That’s not such a big deal, as North American distribution was announced, and other regions are expected to follow, but it’s good to know. See details on the All Things KORG DS-10 blog. (Thanks, DS-10 Dominator!)

Check out some demo videos and a quick run-down on specs, and if you’ve got the capability to run homebrew, you can give this a try. Thanks to Art/toitoy for the tip!

• Two oscillators with fixed oscillators, ADSHR envelopes
• Filter section (in development)
• Ring modulation and “cross” mixing (cross-fading between oscillators)
• 16×16 step sequencer, 4×4 pattern selection

Clearly, there’s a lot of work still to be done, so test this build at your own risk. But it’s already good fun, and the coming roadmap looks logical, with filter options, disk rendering, song mode, and more in store. It’s also an interesting read as far as wrangling with emulators and some of the challenges of DS development.

What happens when you mix technical chatter on the Cakewalk forum, Samuel Beckett, and The Matrix? I’d wager you get something like the surreal video above. Prompted by the posting of technical details for a new update to Cakewalk’s SONAR production software for Windows, and empowered by a strange, new tool that generates eerie virtual reality from typed text, we get banter like this:

The arpeggiator is now on every track, so you are supposed to use it. It is one of the new rules of recording.

Yes, I came from the days of one-finger piano playing. This is a total blessing to me.

I’m going to take that as a challenge and base my review of SONAR 8.5 on using an arpeggiator and step sequencer on every track. And I’ll have to pronounce all those hard g’s in the voice over, clearly.

And no, this is not some twisted viral campaign on the part of the folks of Cakewalk; I’ve been assured that this came from a user.

Okay, what was this post originally about? Oh, yeah – the actual technical details of the SONAR 8.5 release. Noel Borthwick talks about all the details of the new SONAR release on the Cakewalk forums. Apparently, some people care deeply about whether this is SONAR 9 or 8.5 or some conspiracy theory there, but what interests me is the technical details of the software itself.

SONAR 8.5 Fine Print

Noel goes down to a code level. Interesting tidbits: working with Intel, Cakewalk was able to do a demo of SONAR running an absurd number of tracks, instruments, effects, and live video without pegging the CPU, with a tiny 2 ms of latency. The Cakewalk engineering effort also has put together what may be the most highly-optimized VST support and richest 32-to-64-bit bridging on any platform, anywhere.

Whatever the opposite of “marketing speak” may be, I think that’s what Noel has achieved, getting into a sort of developer-to-developer level discussion. It is still readable, and worth digging through.

See also: Intel Developer Forum details and video on the Cakewalk blog

I could talk more about that, but let’s just leave it at step sequencers and arpeggiators on every track, okay?

Windows matters. It’s what roughly half of CDM readers use, and – for all the attention Apple gets – it’s a big part of the computer music world. Windows today also faces many of the same under-the-hood challenges that other operating systems do, so even if you’re a die-hard Linux or Mac user, you may want to pay attention.  You don’t need to love Windows, and you certainly won’t be hosting a Windows 7 launch party. You want to know if the OS will get out of your way and let you get to work.

Windows Vista proved what happens when an operating system’s many interconnected pieces are out of alignment. Even a graphics driver out of sync with underlying changes in the OS could render audio unusable, because just one missed sample can produce an audible glitch or dropout. Part of why I’m optimistic about Windows 7 is that Vista today is a radically different picture, thanks to many, many fixes delivered by Microsoft in updates and more mature audio and video drivers. But that means not just whether 7 is better than XP, but whether 7 is also better than Vista.

Vista wasn’t entirely alone: Mac and Linux have all had their share of growing pains in recent years. The devil is usually in the details. So, I again turn to one of the best guys in the business for sorting out all those technical details. Noel Borthwick, the CTO for Cakewalk, probably has a better big-picture view of how music and audio work in Windows than anyone on the planet. He’s a person hardware and software vendors outside Cakewalk often rely upon as a resource. Noel kept us technically honest on Vista, and he’s doing it again on Windows 7, with some exclusive information for CDM.

Those details get mighty technical, so here’s the punchline: Windows 7 is an OS Noel would use himself. It was hard to get anyone to recommend Vista over XP; loyal Windows-using developers I know still largely stick to XP. But would Noel switch from XP to 7?

Yes, absolutely. Windows 7 finally delivers on the stability and performance that users hoped for from Vista. The kernel changes and optimizations for large scale multi-core processors make it very attractive to DAW users who are interested in better low latency performance. I will be building a new DAW soon and Windows 7 X64 will be my OS of choice.

What’s new in Windows 7?

• Better multithreading: Improved performance of highly-multithreaded software and hardware by removing a significant bottleneck, especially relevant to a tool like SONAR
• Better memory management: Improved memory management when working with multiple threads
• Less nagging: More customization over UAC prompts (meaning they don’t have to nag you more than you want)
• More lightweight: Fewer system services run by default on a stock system, plus a leaner footprint of the OS
• Media support: More native media format support, including QuickTime MOV and H.264, plus drag-and-drop media transcoding
• Composite devices: More logical display of hardware with multiple functions (like audio and MIDI).
• FireWire: Enhanced FireWire support, with IEEE 1394b
• Multi-touch: Multi-touch display support
• Usability improvements: An improved user interface, task bar, and Libraries for managing files

If you’re ready for all the gory details, read on – including a frank appraisal of how all of this compares to XP in real-world performance, and what compatibility issues to look out for if upgrading from either Vista or XP.

Noel Borthwick of Cakewalk effectively wrote this story in response to my questions, so these answers all come from him. Microsoft has not responded to my requests for a review copy, so I’ll be able to evaluate this on my own system – albeit far less scientifically than Noel can – closer to launch.

WARNING: Extremely geeky details of the inner workings of Windows 7 follow, in keeping with our “never dumbed down” policy. If you’re a developer, you can likely get some leads on how to better support Windows 7 in a single point, something even Microsoft doesn’t provide as completely. But if you’re willing to dig, you get a rare view of the OS from a developer view – no marketing speak, no cheerleading, no fanboyism, no platform wars, no writing for the lowest common denominator.

What Actually Improves Audio Performance

Peter: In terms of performance for audio production, what are the significant differences in Windows 7?

Noel: Windows 7 on the surface is very similar to Windows Vista. It has the same audio driver support and same audio system infrastructure as Vista. However, it’s some of the under-the-hood improvements that are more significant for audio production. There are some interesting innovations and optimizations in the Windows kernel, making the OS more scalable for concurrent processing. This makes it attractive for highly multithreaded applications like SONAR. Additionally there are various new API’s/SDK’s that may be of significance to developers. Some highlights are below:

Multi-threading: Removal of the kernel “global dispatcher lock”

In Vista and earlier, on a highly multi-threaded system (e.g. SONAR running on an 8 core hyper-threaded Intel Core I7 PC), you have many threads all processing tiny audio buffers at low latency. All these threads are ultimately waiting on the dispatcher lock when it comes time for them to be managed by the Windows scheduler. This global lock becomes a bottleneck in the system and prevents efficient multi-core workload distribution and scalability. This problem gets magnified as you increase the number of cores since they are all gated by a common lock. In Win 7 the kernel team changed the logic in the Windows scheduler to abolish this global dispatcher lock and use per object locks. This effectively removes this age old bottleneck and allows Win 7 to scale better even under workloads of 256 processors.

This change means a lot to applications like SONAR that rely on multithreaded processing of very small workloads. Initial benchmark results have been promising in this regard. SONAR performs more efficiently at low latency on multi core machines.

Improved Memory Management – PFN database lock

The PFN (page frame number) database lock was used by the memory manager to lock pages of memory in the working set. Like the dispatcher lock above, this would gate memory access from different threads causing resource contention. Work in this was first done in Windows server 2003 SP1 and Windows 7 has now has this optimization as well, improving asynchronous access to memory.

Power Optimization: Core Parking

Windows 7 has a new feature called Core Parking. Core Parking is a power saving optimization that shifts processing load to one or more cores and puts other less busy cores to “sleep”. The objective is to let other cores idle if workload levels allow for it. This optimization had us scratching our heads when we ran a benchmark test on a Quad Core I7 machine. At any point in time, we would notice that some cores were idle in task manager. The reason for this turned out to be Core Parking. Core parking can be useful to save battery life while running projects on laptops.

Better WaveRT Performance

Unlike Windows Vista, Win7 now uses event mode internally. This is good news, since it will help guarantee that HDAudio drivers in Win7 support WaveRT event mode properly. Additionally event mode is now part of WHQL logo certification for driver vendors, so any WAVERT device must support this to get a Win7 compatibility logo.

Ed. note: The plain-English translation here is that WaveRT, Microsoft’s own real-time audio driver facility, now is more likely to work the way you expect. Cockos, makers of REAPER, actually provided the ability to turn off WaveRT Event Mode at the end of last year because of unpredictable results. Windows 7 should resolve these issues.

Other Improvements

Peter: Noel also assembled some other improvements worth noting in Windows 7. They’re subtle, but useful: you may finally be able to avoid installing QuickTime/iTunes just to play some video files, interfaces with audio and MIDI jacks don’t have to show up separately any more, there’s improved FireWire support, usability improvements, and multi-touch on mainstream computers is now nearly here.

Noel:

Additional File Format support

Windows 7 adds native playback support for media in MP4, MOV, 3GP, AVCHD, ADTS, M4A, and WTV multimedia containers. It has native codec’s for H.264, MPEG4-SP, ASP/DivX/Xvid, MJPEG, DV, AAC-LC, LPCM and AAC-HE

Yes you read that right – QuickTime MOV file support is now natively available in Windows 7 so you don’t need to install QuickTime. Another big plus is that this is supported under the X64 version of Windows 7 as well, something you cannot do with Apple’s native QuickTime itself!

All media files using these codec’s should play in Media Player. It appears that these new codec’s are exclusively available to Media Foundation applications and not via other legacy API’s such as DirectShow etc.

File format transcoding

File format transcoding of many popular formats is now built into the Windows 7 shell. I.e. dragging and dropping files onto a device automatically performs the necessary format transcoding if the format is supported. This was primarily done to copy formats to portable devices like cameras but should be useful in other scenarios as well.

Multi-function devices and Device Containers:

Prior to Windows 7, every device attached to the system was treated as a single functional “end-point”. While appropriate for single-function devices (such as an audio interface), this does elegantly represent multi-function devices such as a combination audio/MIDI interface. In Windows 7, the drivers and status information for multi-function device can be grouped together as a single "Device Container", which is then presented to the user in the new "Devices and Printers" Control Panel as a single unit.

http://www.microsoft.com/whdc/Device/DeviceExperience/ContainerIDs.mspx

Note: this should not be confused with device aggregation as is available with Core Audio on Mac OS. On the Mac, you can treat multiple audio interfaces as though they’re one interface, so, for instance, you could get extra outputs by combining a couple of audio interfaces, and your software will see them as if they’re just one box. But SONAR provides this capability on its own, so if you’re a SONAR user, you can get the same functionality.

FireWire/USB

Windows 7 contains a new FireWire (IEEE 1394) stack that fully supports IEEE 1394b with S800, S1600 and S3200 data rates. According to reports, USB 3.0 may be supported in a future Windows Update. It was initially planned for Win7 but is not supported in the shipping version of Win7 due to delays in the USB 3 specification.

Multi-touch

Windows 7 includes integrated support for multi-touch displays.

Libraries

Libraries are user-defined collections of content including folders. It’s a handy way to categorize and create shortcuts to samples, music, etc. Special shell folders (Documents, Pictures, Music, and so on) are now Libraries.

Accelerators for Windows

Windows 7 Accelerators provide a way for learning more about selected text, optionally using voice control.

Virtual hard disks

The Enterprise and Ultimate editions of Windows 7 incorporate support for the Virtual Hard Disk (VHD) file format. VHD files can be mounted as drives, created, and booted from.
An installation of Windows 7 can be booted and run from a VHD drive, even on non-virtual hardware, thereby providing a new way to multi boot Windows.

Leaner Footprint

Win7 has a leaner footprint and has been tweaked to work well on less powerful PC’s, laptops and Netbooks. I have heard reports of Win7 working more smoothly on machines that would be slow under Vista.

Listen Mode

Another nice touch in Win 7 is that they now have a listen tab in the audio properties. Turning on "listen mode" basically routes input to the default output device allowing you to monitor an input device in Windows itself. Sadly this runs via the Windows audio engine which is always running in WASAPI shared mode, so it’s subject to a 30 msec delay. Of course you can always load an application like SONAR and route the audio inputs to an output for low latency monitoring.

Compatibility: What to Watch

Upgrading from Vista

Peter: Relative to Vista, are there any changes that are likely to introduce new compatibility issues with hardware or software?

Noel: With any new OS there is always the potential for compatibility issues. Win7 is built on the Vista foundation and one of its goals was better compatibility. As such most applications that are Vista compliant should work as well or better in Windows 7. UAC in Windows 7 has been improved so this might also help with general compatibility problems with some applications.

We have run into only a couple of compatibility issues in Win7 during the course of our development/testing of SONAR 8.5.

The MMIO API in Win7 (typically used for writing RIFF wave files) has a compatibility issue with the mmioDescend API with LIST ‘WAVE’ chunks. This caused our code that reads audio bundle files to fail and read scrambled audio data. We worked around this problem in 8.5

In WASAPI exclusive mode under Win7, the minimum latency you can achieve is now unfortunately 3ms and the code reports an error if lower. The fact that Vista has no such limitation has been reported to Microsoft. Hopefully its a mistaken fence in their code and this issue is fixed via an update, since it’s a step backwards for low latency in WASAPI mode.

Ed.: That last issue is an interesting one for anyone really pushing the envelope with low latency, so I’ll keep in touch with Noel if there’s any update.

Upgrading from XP

Peter: What hardware and software compatibility issues should users be aware of if they’re thinking of migrating not from Vista but from XP to Windows 7?

Noel: The compatibility issues that typically affect users migrating from XP to Vista/Win7 are:

UAC problems: Many applications and plug-ins are not built to handle the newer security settings in these OS’s. For example, if an application relies on something that requires administrative access it will fail when running as a limited user in Win7. This is a serious issue since in Vista/Win7 even if you are running from an administrator account; programs are launched by default with limited user privileges. Unlike XP, you have to explicitly run as an administrator to use such programs. To be Win7 logo-compatible, all applications need to should support running as a limited user.

Drivers: Although for most practical purposes audio drivers in XP and Windows 7/Vista are similar (you still need to write WDM drivers) there are sometimes quirks in specific drivers may cause problems. Most typical driver issues here are caused by installers that make assumptions about the OS version. In many cases this issue can be solved by the end user by setting the “compatibility mode” to Vista in the file properties for the appropriate driver installer file. (Right click the setup exe file to set its properties)

Ed.: I don’t feel either of these is a deal-killer, as I’ve been living with Vista for some time, but they’re still worth watching out for if upgrading from XP. And it means if you have an older machine that’s still working properly, you’re just likely to leave it on XP and worry about sorting the upgrade on a new box.

Less Nagging?

Peter: We talked when Vista came out about User Account Control and particularly audio-specific tasks that required elevation or different handling of permissions in Vista. I know UAC has been streamlined in W7. Do these changes impact audio apps at all? Are there corresponding under-the-hood changes?

Noel: The UAC changes in Win7 are primarily to allow more customization over the UAC elevation prompting process. There are no changes to the fundamentals of how UAC itself works that I am aware of. The classic problem with audio applications with UAC is when programs or plug-ins write to areas of the registry or file system prohibited from standard user access. Even when you are running as an administrator, by default when you launch a program (or the program itself launches a secondary process) Windows 7 will run that process with standard user privileges. If a program or plug-in attempts to write to an area which it doesn’t have write privileges for, virtualization will kick in. While this may allow the program to work, in general it is bad practice to rely on virtualization, since it can cause many unwanted side effects and behaviors in applications.

There are now four customization settings for UAC:

1. Never notify (least secure). The user is not notified when a program tries to install software or make changes to the computer. The user is not notified when they make changes to Windows settings or when programs try to do so.

2. Only notify me when programs try to make changes to my computer. The user is not notified when a program tries to install software or make changes to the computer. The user is not notified when they make changes to Windows settings. However, the user is notified when programs try to make changes to the computer, including Windows settings.

3. Always notify me. The user is notified when a program tries to install software or make changes to the computer. The user is also notified when they make changes to Windows settings or when programs try to do so.

4. Always notify me and wait for my response (most secure). The user is notified when a program tries to install software or make changes to the computer. The user is also notified when they make changes to Windows settings or when programs try to do so.

Customization and Tuning Advice

Peter: How much customization would you advise people do to their OS? That is, you’ve just installed a build of Windows 7 for working with SONAR on a test machine. Do you run the stock configuration, or start turning off services, disabling disk indexing, etc.?

Noel: Optimization and customization is a topic that can’t be fully discussed in the scope of a brief article. In general you need to optimize a system when you have known bottlenecks. Otherwise you can spend a lot of time tweaking things that have little effect on the end goal. In fact, you may even end up destabilizing a perfectly working system. A stock Win7 machine is not optimized for audio necessarily but it appears MS put some thought into trimming out unwanted startup tasks to cut down on startup time. For example there are now “Triggered start services” in Windows 7, so out of the box you can have fewer services running after a fresh boot. There are probably many background services in a modern DAW that could be suspended if you don’t need them but they should be evaluated on a case by case basis depending on what you use the machine for.

Peter: A lot of users were advising running Vista with Aero off, certainly in the early days. Do you think it’s now advisable to leave Desktop Window Compositing switched on for audio work? (Note: I am aware that there’s actually no way to *completely* disable the Aero windowing environment in a way that it reverts to XP, as even in Class mode with no compositing settings the engine has been altered.)

Generally speaking, turning off Aero will free up some resources on your system, since it uses more costly 3D graphics rendering and transparency a lot. However on any modern graphics card, Aero offloads a lot to the GPU so unless your DAW is also competing for the same GPU resources, turning it off may or may not make an appreciable difference to performance. Most applications that are not graphics intensive use GDI for rendering to the screen and since GDI doesn’t take advantage of DirectX hardware acceleration it’s normally not contesting with the GPU. If you are using plug-ins that use Direct 2D or Direct3D, you are probably better off disabling Aero.

Launch Party, After All?

Thanks, Noel. So, the big news behind all of this is that a move from XP to Windows 7 is finally advisable.

I would still caution, as I did recently with Mac OS Snow Leopard, that you typically don’t want to upgrade to a new OS the day it launches. You’ll want to verify compatibility with your software and hardware before making the jump.

That said, this is an unusual upgrade in that it appears to resolve more issues than it introduces. I actually haven’t been able to find a single user out there testing Windows 7 who has found any issues with audio or music production. Of course, when it launches, we’ll have a much larger test base, so I expect we’ll find something – even Windows Service Packs and point releases of Mac OS have been known to create some issues. As we get closer to launch, I’ll review how you would backup your existing XP or Vista system to ensure that if you do choose to upgrade, you can revert to a previous version.

I am, however, cautiously optimistic. And now is an especially good time to make the jump to 64-bit. It’s easier on Windows than any other OS at the moment, and easiest in SONAR, because SONAR allows you to easily migrate 32-bit plug-ins into the 64-bit environment. You’ll need a 64-bit machine and enough memory to make 64-bit worthwhile, but if you’re building a new workstation, as Noel is, the timing could be perfect.

I also think there’s plenty of room left to talk about issues that go between operating systems, particularly how audio software can better support multi-threading and processing on the GPU, multi-touch, as well as emerging I/O standards like USB3. (OpenCL, much-touted in Snow Leopard, is also supported on Linux and Windows, and Linux actually beat both Mac OS and Windows to the punch in providing a first implementation of USB3.) Correction: I should also add that the excellent Reaper has also added this feature. With full 64-bit support in Cakewalk’s own Dimension and other instruments, NI’s Kontakt sampler, and the bundled 64-bit-native plug-ins in Reaper and SONAR, that means you can build a really capable 64-bit rig on Windows.

With fixes getting the OS out of your way, we can return to issues that really matter, many of which apply to every OS.

Music is, as always, the perfect place to talk about these issues. We push our machines harder than just about anyone, and in ways that are the least tolerant of timing discrepencies and glitches. I’ve said it before, and I’ll say it again: if you want to look into the future of computing, ask a musician.

And that calls for a party.

Previous coverage:

SONAR 8.5 and how it can smooth the transition to 64-bit (8.5 is the build that includes Windows 7-specific improvements)

Vista Tweak: Use the Audio Profile Cakewalk’s CTO Uses

Optimizing for Vista: Inside the Mechanics of SONAR 8 with Cakewalk Engineering

Adieu, XP; How Vista SP1 is Doing, and Why This OS Generation Has Been So Tough

Vista for Audio, 1 Year Later: Talking OS Plumbing with Cakewalk’s CTO

Vista for Music + Pro Audio: Exclusive Under the Hood with Cakewalk’s CTO

And yes, I think Noel deserves an Honorary Contributing Editor position for all he’s done giving us absurdly-precise inside details for how Windows works.

Microsoft product screen shot(s) reprinted with permission from Microsoft Corporation.

Code, hardware, and software: Hack Days are all about getting actual stuff made. Berliners, the Music Hack Day that took place in London at The Guardian now gets underway in Berlin this weekend. For anyone who thought the first event was overly Web-centric, there are some new players in Berlin. Ableton is a sponsor, and Ableton, Native Instruments, and RjDj are all presenting hack sessions (in addition to the more Web-focused / consumer-focused 7digital, Songkick, Cloudspeakers, Mufin, SoundCloud, and Echonest). The awesome German musician magazine DE:BUG is also in on the action. I also see our friends at Future Audio Workshop (developers of Circle) in the lineup.

Check out the details:
http://berlin.musichackday.org/
Weekend schedule

So, German readers, who’s going? I’d love to have some spies tell us what the discussions are with NI and Ableton.

I’m, as always, interested in how we can get past geography and share work internationally. So if you’re doing a project, be sure to take lots of pictures, screen caps, code pastes, and the like, and we’ll feature your work here on CDM.

Future events are planned for other cities, and I hope CDM will be involved in some of them. Boston will be the first US event, but it’s on a date I can’t make it. Anyone have a space here in New York you’d like to suggest?

Rounding up my catch-up-on-iPod/iPhone-stories, here’s one from the developer perspective – one that could face music creation developers on the entire platform.

The Apple iTunes App Store now faces the risk of becoming a victim of its own success. Music applications could be a big part of that, without some adjustments on Apple’s part. The problem is this: incoming music “fan” apps could flood out the music production apps that had enriched the mobile software platform since its debut. I think the need could be really urgent. Consider that part of the appeal of Apple’s mobile platform – yes, even in stark contrast to the Google Android on which I’ve been developing myself – is its spectacular real-time audio tools. Combine that with a disproportionately large number of Mac-using musicians, lots of ingenious apps build on Apple’s Core Audio platform, and we’ve seen a mobile platform with an extraordinary number of tools for music creation.

The problem now is that that unique set of powerful apps could get overwhelmed by essentially unrelated “music” apps. A developer who has asked to remain anonymous is already campaigning for a change. He does a good job of explaining the issue, and what might need to happen to fix it. If you’re a developer, you can add your support and feedback to the idea.

Here’s the full explanation:

Hi Peter,
I am writing you with this email I have been sending other developers on the Apple App Store. Although you probably cannot go visit the apple developer connection website because you dont have a membership, I thought you should be informed about the discussion brewing there. Here it is:

Recently the Music section of the App store has gotten very busy with a new type of app, lets call them Artist Apps or Fan Apps. Some of these Apps are great resources for fans and artists reaching out to their fans and potential fans, and some don’t live up to their potential.

However, they are joining a category that previously moved a lot slower, as the apps that had been populating this category were apps with a lot of development put into them and therefore sold at a higher tier usually- but were released at a slower pace. A look at the top 100 paid music apps illustrates this nicely.

Customers perusing the music section to catch that next great sound generating tool (for example), could check in on the new releases section perhaps once a week or even once a month and have the opportunity to check out all the great new apps that had been released, without worrying that one was missed.

Now however, these newer Artist apps have flooded this category, and great apps are getting lost in the shuffle. On one day last week, there were 21 pages of Artist or Fan apps, with a few “other” apps strewn in the mix here and there, very hard to pick out of the jumble.

I understand that this may be happening in other categories for other reasons, but I only concentrate on the Music section since I am a music producer and music App writer.

I propose that we all get together to come up with some suggested sub-category names for the music category. I will start the list off and hopefully some of you will chime in and give suggestions for other categories or add more definition to a sub-category that is alredy here.

Once enough input is received, I will compile it into one bug report for Apple. i will then post the bug# for everyone to include with any correspondance with Apple on this issue.

New sub-categories for the Music section of the App store.

Music Creation:
Synthesizers, drum machines, sound generators, scoring and notation, sequencers, DJ apps, recorders (multi track)

Music Utilities:
Lyrics apps, iPod interfaces, visualizers, iPod controllers, song recognizers, concert finders,

Learning:
Metronomes, guitar and voice tuners, music slow downers, guitar tutors, chord apps,

Artist Apps/Fan Apps:
iLike apps, Deadmou5 app, PVD App, Underworld App, NIN, etc.

Radio Tuners:

AOL Radio, Pandora, Last.fm, individual radio stations

Please visit the Apple iPhone developer forums and voice your opinion/support!

https://devforums.apple.com/message/107989#107989

Definitely voice your thoughts to Apple, but I’d love to hear what you think here on CDM, too (especially since I know some folks at Apple do read this site).

Well, fine, Jamie Lidell. Now you go and ruin it for the rest of us. See, none of us playing with a Nintendo DSi will possibly look as good as you do.

I jest, of course. Jamie Lidell, the wildly-talented vocalist, picks up the new, online-savvy take of the Nintendo DS and breathes cool into it. This is what Sony ads tried to do, but Jamie does masterfully. And, okay, don’t expect the built-in sound app on the DSi to do as much as it appears to be doing here – there’s quite a lot of non-real-time, non-DSi remixing going on, even though what he does do with the simple app is genius.

Thanks to Liz Revision for finding this one.

This does bring us to a burning question: Nintendo and Sony, I’m looking at you. When will we be able to run eccentric and niche music creation apps as official software on your machine? Imagine NitroTracker on the DSi download store or PSPSEQ and PSPRhythm on the Sony Store.

It’s not lost on me, either, that I’ve complained about Apple’s App Store approval for iPhone and iPod touch as being vague and inconsistent with a few apps, while Nintendo and Sony and Microsoft’s Xbox only allow a few apps to ever see distribution. But now that the game makers have online distribution, I wonder if that could change. Movie multiplexes once promised that, amidst a few dozen screens, one would show Bollywood and experimental film. Perhaps these stores could have an “enter at your own risk” category for homebrew. Sony, after all, is desperate to recover sales lost on its PSP. And I have to admit, I think homebrew is, sadly, partly at fault. Homebrew developers and users painstakingly document hacking steps because it’s the only way to get their software on the device – only to have the same system abused by people who don’t want to pay for games.

Korg came out with its DS-10 app for the Nintendo handheld officially, and it was an enormous, runaway success, spawning YouTube virals and entire bands. Now, granted, the app had a major commercial publisher behind it (AQ Interactive), adding credibility – but distribution was limited by the physical cartridge, and the app itself didn’t shy away from Big Boy, niche soft synth controls.

8-bit musicians will likely never touch the DSi, preferring the vintage Game Boy. But a few hard-core gamers are also hard-core handheld musicians. Underground is great, and there’s a certain ethos around hacking. But access isn’t such a bad thing. If just one advocate at Sony or Nintendo would consider it, I think wonderful things could happen.

Now, if you’ll excuse me, I have to see if it’s possible to get a DS homebrew music app running in an emulator on my Android.

“Augmented Reality” is a fancy term for describing ways of using computer vision to overlay digital intelligence on images. In other words, you can, for instance, scratch a vinyl record using a camera – plus a tag for identifying the object’s position in 3D space.

Cambridge-based designer Todd Vanderlin put together an elegant demonstration of the possibilities here, and his video has accordingly been making the rounds. (See: Synthtopia – and I actually heard about it this morning from a high school friend. The power of the Internet.)

Todd has more details on his site, which includes all kind of wonderful projects, like laser sound fountains and, always favorite around here, creepy circuit-bent baby dolls.

AR Scratching [Todd Vanderlin]

There’s actually some work to this: you need to figure out how the album is spinning. And of course, because this is augmented reality and not reality, there’s real potential here to imagine a new kind of vinyl DJing in which normal physics don’t apply.

From the video description:

I was playing around with some AR markers the other day and came up with this idea. taking just a plain old vinyl record and attaching an AR marker to the label you can track the record in 3D space. The next question was, can you scratch the record?

So by figuring out the velocity of the records rotation and applying it to the payback of the audio you can scratch. There is some digital noise that needs to bee worked out, but sounds pretty good. Its still really hard to scratch, it takes some practice but is super fun. The next step is to figure out some nice triggers for different modes. I like the idea of not needing a turntable but the actual spinning of the record helps with the scratching and playback. I made a couple modes, one where the record is paused and you can just scratch through the song. The other looks for zero velocity for x time and then continues on with the song. If there is velocity you then are scratching and the audio is affected. I think that this project has some legs can’t wait to play more.

I Want My Augmented Reality TV

So, this has sufficiently inspired you and you want more augmented reality? We’ve got more for you.

Digital artist and magician Marco Tempest has just demonstrated what happens when you do card tricks with augmented reality – and he shares some details of his rig:

Virtual Magic: Augmented Reality Card Tricks with Marco, OpenFrameWorks [Create Digital Motion]

We’ve even seen augmented reality climbing walls.

If you’re ready to do this yourself, we have a number of resources:

CDMotion has Andy Best’s tutorial on OpenCV with Processing (not an AR-specific library, but relevant): Getting Started, Popping Bubbles

We’ve also got an in-progress library for use with mapping projections in space for “spatial augmented reality”

A New Year’s video with a library for Flash, Java

A tutorial on getting started with augmented reality using Flash

openFrameworks, a library for C++ coding (which in turn supports multitouch, augmented reality – you’ll see some projects on that page)

Bryan Chung is working on a library for Processing

Take a powerful game engine (for animation, 2D and 3D graphics, physics, and on-screen interaction). Add the flexibility of a visual development environment for programming with virtual patch cords, for rich sonic and musical capabilities plus easy interaction with data and input. That’s the idea of combining something like Unity 3D with Max/MSP. In the example from earlier today, the solution simply routed basic data from a Unity-based game to a responsive music engine in Max.

In the case of [myu] – the Max Unity Interoperability Toolkit – that integration goes further still. Developed at the DISIS (Digital Interactive Sound & Intermedia Studio) at Virginia Tech, [myu] allows bi-directional integration of the Unity engine with Max or Pd. The two tools use netsend/netreceive to send data via TCP and glue the two together.

For visualists using Jitter, you can even exchange texture data, which offers some mind-blowing powers for live visuals.

Download at Virginia Tech — bonus, an extension of the aka.wiiremote object so you can use the lovely Wii Fit controller, among various other projects
Discussion on the Unity Community Forums
Discussion on the Cycling ‘74 forum
Virginia Tech DISIS

As an interactive prototyping tool, this should have a lot of potential for lovers of patch-style programming.

Thanks to Dr. Ivica Ico Bukvic, DISIS Director and researcher, for sending in his project. I’ll be curious to see what other people might do with this.

A real highlight for me at the Game Developer Conference was getting to hear Satoru Iwata deliver the keynote. Aside from being CEO of Nintendo as they have launched their most successful console ever, Iwata-san has left a sizable development legacy as a veteran of HAL Laboratory (Balloon Fight, Kirby). In the game community, I think the reception to his keynote was mixed – mostly, it introduced long-overdue storage solutions for Wiiware titles, along with some relatively minor game titles. But as a person interested in design and development – and what innovative interfaces could do for music and not just games – I found the rare insight into Nintendo’s development process inspiring.

The surprise: despite their enormous resources, Nintendo is moving to ever-smaller development teams. And they’re taking dance classes to work on their musical rhythm.

Any developer with limited resources is familiar with what Iwata described as the “development death spiral”: financial pressure means rushed titles with poorer quality, resulting in fewer sales, resulting in greater financial pressure. Oddly, Iwata didn’t quite explain how do navigate out of the death spiral, explicitly. “Once you enter the death spiral, it is difficult to escape,” Iwata acknowledged. But the implication of his presentation was that you could do more with less, by focusing on process – not necessarily adding resources, but focusing on humans and fun. (The analog for music, perhaps, would be as much “expressivity” as fun.)

To illustrate, Iwata spoke mainly of Nintendo’s chief designer, Shigeru Miyamoto, creator of … um, nearly everything, in fact. Now, some of Miyamoto’s habits have been widely published, like his tendency to turn hobbies (gardening and puppies) into games. But to hear a normally-secretive Japanese company talking frankly about its process is something special.

Miyamoto’s Way is what Iwata called an “Upward Spiral.”

He observes people everywhere having fun. He thinks about how the core of that fun might come into games. But even as one project starts, he is still observing other people having fun – more ideas are born. Most developers prepare a thick design document to explain their intension to their teams. Mr. Miyamoto almost never writes one.

His first goal is always the same – a [prototype,] very limited and very clear. The amount of time being spent on the game’s appearance is zero.

Mr Miyamoto always has multiple projects in this stage at the same time.

What I find most important is how in each phase of overall development, he can clearly distinguish which details must be perfectly finished in that phase, and separate them from the parts that can be tentatively prepared.

You can see just how primitive some of these prototypes are in the example below from Punch Out. I think this is actually an important issue, as many beginning developers of games and audiovisual works don’t get primitive when doing early drafts, thus making it harder to make changes later.

Miyamoto is also notorious for randomly kidnapping employees for playtesting – playtesting without focus groups or statistics collection, but more qualitative evaluations of how people like a creation. Again, this isn’t unheard of in the industry, but it seems not to happen enough. And Miyamoto looks very fetching in his Cowboy / Outlaw getup.

Now, prototyping is nothing unique to Nintendo. But remarkably, Miyamoto’s prototype phase can last “more than two years.” And while no other developer has Miyamoto working for them, I expect that this is unusual:

I make it a point not to ask how [the project]’s doing. I believe this could make the team cut corners, or settle for less than their desired outcome.

This is not very good for my mental health. This is because of Mr. Miyamoto’s tendency to … upend the tea table.

“Upending the tea table” – also known as the “Miyamoto Test” – is a signature Miyamoto move by which the designer scraps a development process in mid-stream in order to make corrections. Again, this happens in the game industry, though perhaps not as often as it should – and certainly, no one has the leeway Miyamoto does.

Mr. Miyamoto is never an … angry man. He resets the dishes he had scattered, explaining just how they should be arranged on the tray.

I know many developers and critics are increasingly becoming frustrated with the dogma of fun, believing it forces the industry into a narrow range of expression. But, then, I enjoy depressing movies. Defined as enjoyment, Nintendo’s philosophy of fun is more a kind of commitment to its users. As Iwata puts it:

We create entertainment, and entertainment is meant to be enjoyed. If it can’t be enjoyed, it’s not the consumer’s fault – the fault belongs to us.

(At this moment in the presentation, in fact, Iwata bent forward slightly and halted, as if to consider the shame of such a potential situation.)

The musical connection to all of this is the rhythm game, “Rhythm Heaven.” We were lucky enough to get a copy for DS as we left the presentation; more on how it works soon. The game has already had a life as a Japanese-only Game Boy Advance title, but is now a worldwide release on DS. Several revelations were interesting to me in this presentation:

• Nintendo is turning to increasingly-smaller teams – as few as five on the GBA game and three on the DS. That says a lot about the way the videogame titan views effective development, and should give hope to penny-pinching indie developers and publishers, as well as us musical / visual experimenters toying with developing new interfaces.
• Rhythmic theory: The impetus for the game was designer/developer Tsunku’s new “rhythmic theories,” and ideas about how to teach and play with rhythm.
• Dance instruction: To help developers learn better rhythm themselves, Nintendo turned not to music lessons but dance movements – Tsunku bet that movement would help hone the programmers’ rhythmic skills. 

As Iwata explained:

How can a good rhythm game be created if the developers’ themselves don’t have much rhythm?

The quickest way to learn rhythm, [Tsunku] believes, is to dance. So the developers danced. Maybe they’re like winners of the Japanese ‘Dancing with the Stars.’

This was the first time as a game producer that I had to approve a budget for dance lessons.

For those of you wondering what the future of platforms for gaming or music are, Iwata had other juicy stats, as well. In 2008, female usage of the DS was up sharply to 47%. That brings hope for less male dominance of music technology. And anyone betting the iPhone would obliterate the DS as a gaming platform ought to think again. The DSi – the latest DS model with downloadable titles and a built-in camera – set a new advance-order record on Amazon for game systems. Some 90% of WiiWare titles are independent, so that makes me imagine that we could see creative new music and visual creations on both WiiWare and the DSi download service soon – a nice change from the current situation, which requires you to hack your system just to get real music apps. It’s nowhere near as open as the iPhone, though, so installed base aside, I think the iPhone / iPod touch remains a friendlier development platform.

Iwata closed with a nice sentiment for all of us:

Remember, in the Great Depression American inventors invented the jet engine, television, and even the chocolate chip cookie. As a developer, I believe anything is possible. The future of video games is in your hands.

I’m always a fan of “off-the-fovea” thinking, which was part of why I went to GDC. Hearing game developers tackle these problems I think has a lot of lessons for development of other creative projects – and I certainly believe a lot of these lessons are applicable to audiovisual makers, even if you don’t intend to release an iPhone – erm, DS — music game. Prototyping, testing and observation, small teams, using movement to make music and rhythm more powerful – all of these have great lessons not only relative to the game industry’s norms but for everyone else, too. I’m curious to hear what you think. But, if you’ll excuse me, I’m going to take a DS break.

The “library access” in the 3.0 SDK is only a player control API similar to that of the iPod; there is not even read only file access for MP3s nor any way to modify the output from the library. So no iPhone DJing, no BPM detection, no interactive PD or Reaktor patches with your library.

Unfortunately, this seems consistent with Apple’s desire to be the one and only media player on the device. I’m hoping that this is still something Apple plans to add – imagine the ability to add effects or run games based on the library (a la the PC game Audiosurf) or create DJ apps. I know many people who use iPhone or iPod as sample players or backups for live sets; having a custom player app could also be useful.

By comparison, Google’s Android has no such limitations on its MediaPlayer class – the fundamental difference being that you aren’t limited from playing media on your device. Unfortunately, Android has its own limitations: no real audio buffer access, which means it’s not possible to build effects or DJ apps or games on Android, either.

And that’s typical of the sort of situation the newest mobile devices present. We have the iPhone, more sophisticated technically, but limited, apparently, by design in order to protect Apple control over certain functions. Then we have the Android, philosophically unlimited but technically limited by some key missing capabilities.

My question is, which device will evolve first to give us the freedom to make use of its full potential?

No file or output access to iPhone MP3 library – 3.0 SDK still too restrictive

If we’re lucky, perhaps the 3.1 SDK? (Or something we’ll still see in 3.0 that isn’t done yet?)