Yesterday, I talked about two complaints of music developers writing applications for the iPhone. These come from developers who are really iPhone fans, who just want to get their software released and (for many music devs) better categorized on Apple’s store. Pajamahouse Studios, maker of the new Sonorasaurus remix application, follow up with a more detailed explanation of their situation.

These are not rejections; at least rejections are generally accompanied with some sort of suggestion of what would need to be changed. They represent the dreaded iPhone developer “limbo,” in which an application is neither rejected nor approved. For Sonorasaurus, that’s been the state of affairs for over two months. As the developers explain, there seems to be nothing unusual about their app:

• Library access: It doesn’t access the iPhone/iPod music library. (no application is allowed to do that, which incidentally limits a lot of the DJ app possibilities of the device) Clarification: The status of the music API itself is unclear; some developers report just this sort of approval delay when trying to use it. [Source] Also, access to files inside the media library is not directly possible, which can be compared to the status of Android.
• File access: A separate http server is provided, with a parallel library, for users to store their own tracks – again, something found on numerous other approved applications. This doesn’t use the included library.
• Included music / music distribution: Five included songs are for testing only – something found in a number of other, similar applications that have been approved. The application is not an alternative to iTunes for distribution.
• Media decoding: Custom MP3 decoding technology – something not provided on the iPhone – was separately licensed. Clarification: This was not meant to imply that you can’t do MP3 decoding; the developers meant to make the point that they were not violating patents or licensing by using their own decoding, which presumably they did for the purposes of building a DJ app.

Of course, whatever the reason, we’ve seen in past applications suddenly approved after weeks or months, so who knows what will actually happen with this app.

Read the full explanation:

In Limbo Pt. 1 [Sonorasaurus]

While reading that, though, I also have to observe how significant these workarounds are. Without launching into an Android versus iPhone debate – believe me, there are many, many things to criticize about the Android as a platform, especially relative to music –  none of these is an issue on the Android. Forget platform wars or fanboys. Alternatives are good. I’d hope that we do have more than one approach to how to do this. These approaches should have to compete with one another, as they offer different tradeoffs and advantages.

If music is becoming an application, this kind of freedom is important.

Point by point:

• Library access: Android’s standard, supported APIs provide access to the media library and user playlists. Clarification: this includes direct access to the files and the ability to read from the buffer of these files (with some effort), in public, documented, approved APIs, with no chance of having an app rejected for the use of these APIs. My understanding is that this is not exactly the case on the iPhone.
• File access: Users are free to put files on their SD card over USB, and off-load those files – neither possible on iPhone. And yes, these will be integrated with the media library; iTunes-style sync isn’t necessary.
• Included music / music distribution: Including songs is actually a bit of a challenge, but you can freely download content and store it on the SD card. Because Google doesn’t have an equivalent of iTunes, that includes creating your own alternative distribution methods – meaning a label or music store can do make their own outlet.
• Media decoding: Decoding technology is included on the phone, including the ability to decode the open OGG Vorbis format. Clarification: Some folks read this to mean that the iPhone can’t decode MP3s, which was not what I intended; the key point here is that Android has in-box support for free formats and byte-level access to the audio buffers they give you, by default, straight out of the user’s media library. That is not entirely the case on iPhone.

Beneath all of this is the major fundamental difference, which is that you can install applications for Android whether or not they’ve been approved for Android. There’s actually a checkbox in the Market that allows you to opt into installing other apps you’ve downloaded directly from a developer or from another source.

Again, I don’t mean to make a pro-Android argument. In fact, I believe many of these items are also true on Windows Mobile, Symbian, and upcoming Linux platforms; I just happen to be working on Android now, so I’m more familiar with it.

What’s important is that this represents an alternative approach to how to provide music as an application, one in which the user is free to load content on and off the device.

Specifically, this paragraph jumped out at me:

Another problem would be that Apple could see this as a means to circumvent iTunes as a means to sell and distribute music. This we also addressed. These songs can only be used within the App. They can not be removed from the app / device for use elsewhere (iTunes on the desktop, burning to a CD, etc).

Now, we hear from many developers that the iTunes integration is something that attracts them to the platform. Likewise, many content creators will want just these sorts of restrictions.

But what if you want fewer restrictions? Let’s say you’re an artist releasing Creative Commons-licensed tracks, and you want to encourage remixing, sampling, modification, and free use of your tracks. Or what if you’re a label or artist collective, and want to experiment with new ways of using mobile for distribution, beyond what’s possible with iTunes and Apple’s stores? The same qualities that may attract someone else should, I think, concern you. I don’t think that necessarily means you shouldn’t write an iPhone application with your music, but perhaps you should also consider trying an alternative platform.

There seems to be a growing sense that the iPhone Way is The Only Way. Obviously, that’s not the case. This very debate demonstrates just how much room for interpretation the distribution of content can produce.

A powerful DJ application for your iPhone or iPod touch may be a tantalizing prospect. But several would-be candidates aren’t available to you yet. Why? They’re languishing in Apple’s approval process, with no sign of whether they’ll be released or not.

For all the success of Apple’s App Store, some developers and users continue to express frustration at what they believe is a sluggish, unpredictable approval process, restrictive Apple policies, and Apple’s complete control over distribution and categorization. That now leads to two complaints from music developers. A number of music developers want more delineation from Apple’s categories, so that the flood of general music apps don’t drown out powerful, creative tools. Meanwhile, developers of DJ applications claim that Apple is discriminating against DJ apps, which they say has led to delays without explanation.

“Open” development is relative, without question. Game system makers require developers to prove to them why they should be “allowed” to create titles, leading to a tightly-controlled stream of approved titles. But the success of Apple’s relatively open development model has prompted many software creators to hunger for greater freedom. I’ve increasingly heard people enthusiastic about the more flexible distribution model on Google’s Android (and other Linux) platforms, which allow users to install apps they want. I even moderated a mobile music platform panel at the CMJ music conference at which a Verizon representative, no less, talked about wanting to be more open to applications. The benchmark was Apple, for being perceived as overly restrictive.

iPhone/iPod touch developers, however, aren’t simply ranting against Apple. They’re complaining because they’re enthusiastic about the App Store. They want changes from Apple and believe there’s potential to get what they want. That said, I think they also illustrate potential for rivals like Google to outdo Apple – assuming those rivals invest more time and effort into courting these kind of applications.

Is Apple Blocking DJ Apps?

First, some developers believe that Apple is intentionally blocking DJ applications from being approved. Whether intentional or not, a number of potentially ground-breaking applications are unavailable after a significant delay. Kasabian Kasabianmeister writes:

Apple is deliberately not allowing DJ apps to the App Store

Something really strange is going on with the Apple review team. They now seem to approve all kinds of applications, even the ones that have been previously considered "unacceptable", but there is one kind of applications that are simply kept "in review" stage for months without any explanation.
http://www.youtube.com/watch?v=tVv4PY6st6Y

These are the DJ applications that have been developed with the idea to give the user the ability to mix his own MP3 tracks on the iPhone. Currently, there is no application in the AppStore that has such functionality. Of course, this wasn’t left unnoticed by the developers, but…

At least 3 applications: Touch DJ (www.amidio.com), Sonorasaurus (www.sonorasaurus.com), DJ Player (www.djplayer.fm) are not being approved since the beginning of September, hitting the 2-month "in review" mark. One of the developers even made a video voicing the frustration over the absolutely unacceptable behaviour of Apple:
http://www.youtube.com/watch?v=jDUS_2rcAjw

What is really weird and unprofessional, Apple doesn’t give any reasons whatsoever what is the reason of such delays. The developers are just told to "wait" without any explanations.
Meanwhile, the demand for the DJ apps is so high that people even started an online petition entitled "Apple, Allow DJ apps on the iPhone!":
http://www.petitionspot.com/petitions/iphonedj

One of the reasons for such attitude could be that Apple is working on its own DJ app, or is waiting for a DJ app from a "senior" player and keeping the possible "competitors" aside. In any case, it is quite possible that we will know the real reason soon.

It’s worth noting that, in the past, Apple’s application process has simply proved to be inconsistent and slow, which can cause people to see intention where there is none. But that doesn’t necessarily excuse Apple’s App Store approval process. The iPhone and iPod touch are popular largely because of Apple’s success at making media playback devices. Apple needs to document what it views as acceptable use of these devices. In the absence of information, developers are jumping to their own conclusions, possibly accurately, possibly not.

Obviously, aside from interest in Apple’s policies, I’m sure many iPhone and iPod touch owners are eager to see these applications, so we’ll certainly be following them to see if any are released.

In other rejection news… The App Store is currently just letting these DJ apps languish in limbo rather than providing a rejection. But another significant set of rejections is game titles built with the Unity game engine. There, the issue appears to be the presence of support in the engine for a private API call from Apple, whether or not the title itself uses the API, and …uh, yeah, that one’s complicated. Updated TUAW tells us via Twitter that Unity developers say they’ve fixed that issue, so… move along, nothing to see here.

Do Music Apps Need Better Categorization?

One potential danger of having a centralized store like iTunes and the App Store is giving control of approval to a single company (Apple), as seen in the DJ apps. The other is that such a storefront will simply not be categorized in a way that allows people to discover apps successfully.

Apple has been roundly praised for creating a store that encourages people to consume apps. Now, some developers want Apple to tweak their categorization to allow some of the most creative applications to stay in the forefront. Over the summer, Jokton Strealy, maker of the excellent songvoo music collection management tool, issued a call to fellow developers to try to get better categorization from Cupertino.

Side note: songvoo is just the sort of app that critics of the App Store might assume would be impossible. It replaces the existing playback functionality of the iPod and iPhone, the sort of replacement app that has sometimes earned rejection from Apple. Evidently, if an application is differentiated enough, it can clear Apple’s approval hurdles. (On the other hand, inconsistent policy and overzealous restrictions are at the center of some of those criticisms.)

But Strealy has no complaints about the App Store itself or the approval process. He just wants a more intelligently-organized store. Here’s his open letter from the summer, which has since earned a lot of support from fellow developers:

Recently, the Music section of the App store has gotten very busy with a new type of app — let’s call them Artist Apps or Fan Apps. Some of these Apps are great resources for fans and artists reaching out to their fans and potential fans, and some don’t live up to their potential.

However, they are joining a category that previously moved a lot slower, as the apps that had been populating this category were apps with a lot of development put into them and therefore sold at a higher tier usually- but were released at a slower pace. A look at the top 100 paid music apps illustrates this nicely.

Customers perusing the music section to catch that next great sound generating tool (for example), could check in on the new releases section perhaps once a week or even once a month and have the opportunity to check out all the great new apps that had been released, without worrying that one was missed.

Now however, these newer Artist apps have flooded this category, and great apps are getting lost in the shuffle. On one day last week, there were 21 pages of Artist or Fan apps, with a few "other" apps strewn in the mix here and there, very hard to pick out of the jumble.

I understand that this may be happening in other categories for other reasons, but I only concentrate on the Music section since I am a music producer and music App writer.

I propose that we all get together to come up with some suggested sub-category names for the music category. I will start the list off and hopefully some of you will chime in and give suggestions for other categories or add more definition to a sub-category that is already here.

Once enough input is received, I will compile it into one bug report for Apple. i will then post the bug# for everyone to include with any correspondence with Apple on this issue.

New sub-categories for the Music section of the App store.

Music Creation:
Synthesizers, drum machines, sound generators, scoring and notation, sequencers, DJ apps, recorders (multi track)

Music Utilities:
Lyrics apps, iPod interfaces, visualizers, iPod controllers, song recognizers, concert finders

Learning:
Metronomes, guitar and voice tuners, music slow downers, guitar tutors, chord apps,

Artist Apps/Fan Apps:
iLike apps, Deadmau5 app, PVD App, Underworld App, NIN, etc.

Radio Tuners:
AOL Radio, Pandora, Last.fm, individual radio stations

Please visit the Apple iPhone developer forums and voice your opinion/support!
https://devforums.apple.com/message/107989#107989

Sincerely,
Jokton Strealy., President
Elements2Dance.com

If nothing else, the explosion of development for iPhone and iPod touch is prompting some lively discussions about just what development should look like. A lot of what you hear is praise for the Apple model, but I expect some of the criticism of it – even down to minor details – could be productive, as well. I’ll certainly be watching the development of both of these issues, and we’ll see if Apple responds or not.

Updated: Jokton notes an additional change to the way apps are listed.

As for my original issue with categories and release dates, there has been a new development. Apple recently stopped listing “updates” in the “release date” listing of each category. Now the only way to get a listing there (which generates sales) is release a new app (1.0). Before that, devs were releasing constant updates to an app to keep it “on the radar” on the app store, even if the update were a simple as correcting a spelling error or perhaps even faking issues to correct. That in itself created a lot of “update spam”. Many developers are up in arms about these new changes because previously this was the only way to get your app seen by the masses. There has been no official word from Apple on the issue, so it is still unclear if this is a permanent change or some kind of error in the system. If this change is indeed permanent, then the argument for more subcategories is even more important now.

Commercial developers are now releasing music creation apps for mobile game systems, in the form of the KORG DS-10 for Nintendo DS and Rockstar’s Beaterator for PSP. But some of the best ideas still come from the homebrew community.

What’s most impressive about ToneSynthDS is not so much what it does as its interface, fitting all its functionality into the DS’ two compact screens. Its minimal interface finds an elegant arrangement of everything you most urgently need, with a sequencer screen on one DS screen and basic virtual analog synth parameters on the other. A 4 x 4 matrix next to the main sequencer grid lets you switch between patterns, in a step sequencer reminiscent of the monome and Tenori-On. There isn’t a whole lot of depth to event editing in this early version, but it could be a lovely way to sketch melodic patterns. (And some of those limitations come from the DS itself. Note, though, that this app gets a full 16 real-time channels on the original DS hardware to the Korg DS-10’s paltry two.)

Developer Fanta/Hotelsinus Sound Design has been posting mock-ups, demos, and now builds as he goes. That means that he gets feedback from an audience of readers and incorporates those as he develops the app – another key difference between the DIY/homebrew scene and conventional commercial development.

More good news: this DS app should also run as a PC VST in a forthcoming version, opening up the fun to folks using netbooks and laptops instead of the DS and creating a nice mobile-to-computer workflow.

http://ndscomposer.blogspot.com/

In related Nintendo DS news: If you’re thinking about getting the new DS-10 Plus Limited Edition of the KORG DS-10, you’ll need to get it for the region coding of your DS. (In other words, you probably won’t want to import it.) The “Dual Mode” functions are region-locked, so North American and European users can’t use the Japanese DS-10. That’s not such a big deal, as North American distribution was announced, and other regions are expected to follow, but it’s good to know. See details on the All Things KORG DS-10 blog. (Thanks, DS-10 Dominator!)

Check out some demo videos and a quick run-down on specs, and if you’ve got the capability to run homebrew, you can give this a try. Thanks to Art/toitoy for the tip!

• Two oscillators with fixed oscillators, ADSHR envelopes
• Filter section (in development)
• Ring modulation and “cross” mixing (cross-fading between oscillators)
• 16×16 step sequencer, 4×4 pattern selection

Clearly, there’s a lot of work still to be done, so test this build at your own risk. But it’s already good fun, and the coming roadmap looks logical, with filter options, disk rendering, song mode, and more in store. It’s also an interesting read as far as wrangling with emulators and some of the challenges of DS development.

What happens when you mix technical chatter on the Cakewalk forum, Samuel Beckett, and The Matrix? I’d wager you get something like the surreal video above. Prompted by the posting of technical details for a new update to Cakewalk’s SONAR production software for Windows, and empowered by a strange, new tool that generates eerie virtual reality from typed text, we get banter like this:

The arpeggiator is now on every track, so you are supposed to use it. It is one of the new rules of recording.

Yes, I came from the days of one-finger piano playing. This is a total blessing to me.

I’m going to take that as a challenge and base my review of SONAR 8.5 on using an arpeggiator and step sequencer on every track. And I’ll have to pronounce all those hard g’s in the voice over, clearly.

And no, this is not some twisted viral campaign on the part of the folks of Cakewalk; I’ve been assured that this came from a user.

Okay, what was this post originally about? Oh, yeah – the actual technical details of the SONAR 8.5 release. Noel Borthwick talks about all the details of the new SONAR release on the Cakewalk forums. Apparently, some people care deeply about whether this is SONAR 9 or 8.5 or some conspiracy theory there, but what interests me is the technical details of the software itself.

SONAR 8.5 Fine Print

Noel goes down to a code level. Interesting tidbits: working with Intel, Cakewalk was able to do a demo of SONAR running an absurd number of tracks, instruments, effects, and live video without pegging the CPU, with a tiny 2 ms of latency. The Cakewalk engineering effort also has put together what may be the most highly-optimized VST support and richest 32-to-64-bit bridging on any platform, anywhere.

Whatever the opposite of “marketing speak” may be, I think that’s what Noel has achieved, getting into a sort of developer-to-developer level discussion. It is still readable, and worth digging through.

See also: Intel Developer Forum details and video on the Cakewalk blog

I could talk more about that, but let’s just leave it at step sequencers and arpeggiators on every track, okay?

Windows matters. It’s what roughly half of CDM readers use, and – for all the attention Apple gets – it’s a big part of the computer music world. Windows today also faces many of the same under-the-hood challenges that other operating systems do, so even if you’re a die-hard Linux or Mac user, you may want to pay attention.  You don’t need to love Windows, and you certainly won’t be hosting a Windows 7 launch party. You want to know if the OS will get out of your way and let you get to work.

Windows Vista proved what happens when an operating system’s many interconnected pieces are out of alignment. Even a graphics driver out of sync with underlying changes in the OS could render audio unusable, because just one missed sample can produce an audible glitch or dropout. Part of why I’m optimistic about Windows 7 is that Vista today is a radically different picture, thanks to many, many fixes delivered by Microsoft in updates and more mature audio and video drivers. But that means not just whether 7 is better than XP, but whether 7 is also better than Vista.

Vista wasn’t entirely alone: Mac and Linux have all had their share of growing pains in recent years. The devil is usually in the details. So, I again turn to one of the best guys in the business for sorting out all those technical details. Noel Borthwick, the CTO for Cakewalk, probably has a better big-picture view of how music and audio work in Windows than anyone on the planet. He’s a person hardware and software vendors outside Cakewalk often rely upon as a resource. Noel kept us technically honest on Vista, and he’s doing it again on Windows 7, with some exclusive information for CDM.

Those details get mighty technical, so here’s the punchline: Windows 7 is an OS Noel would use himself. It was hard to get anyone to recommend Vista over XP; loyal Windows-using developers I know still largely stick to XP. But would Noel switch from XP to 7?

Yes, absolutely. Windows 7 finally delivers on the stability and performance that users hoped for from Vista. The kernel changes and optimizations for large scale multi-core processors make it very attractive to DAW users who are interested in better low latency performance. I will be building a new DAW soon and Windows 7 X64 will be my OS of choice.

What’s new in Windows 7?

• Better multithreading: Improved performance of highly-multithreaded software and hardware by removing a significant bottleneck, especially relevant to a tool like SONAR
• Better memory management: Improved memory management when working with multiple threads
• Less nagging: More customization over UAC prompts (meaning they don’t have to nag you more than you want)
• More lightweight: Fewer system services run by default on a stock system, plus a leaner footprint of the OS
• Media support: More native media format support, including QuickTime MOV and H.264, plus drag-and-drop media transcoding
• Composite devices: More logical display of hardware with multiple functions (like audio and MIDI).
• FireWire: Enhanced FireWire support, with IEEE 1394b
• Multi-touch: Multi-touch display support
• Usability improvements: An improved user interface, task bar, and Libraries for managing files

If you’re ready for all the gory details, read on – including a frank appraisal of how all of this compares to XP in real-world performance, and what compatibility issues to look out for if upgrading from either Vista or XP.

Noel Borthwick of Cakewalk effectively wrote this story in response to my questions, so these answers all come from him. Microsoft has not responded to my requests for a review copy, so I’ll be able to evaluate this on my own system – albeit far less scientifically than Noel can – closer to launch.

WARNING: Extremely geeky details of the inner workings of Windows 7 follow, in keeping with our “never dumbed down” policy. If you’re a developer, you can likely get some leads on how to better support Windows 7 in a single point, something even Microsoft doesn’t provide as completely. But if you’re willing to dig, you get a rare view of the OS from a developer view – no marketing speak, no cheerleading, no fanboyism, no platform wars, no writing for the lowest common denominator.

What Actually Improves Audio Performance

Peter: In terms of performance for audio production, what are the significant differences in Windows 7?

Noel: Windows 7 on the surface is very similar to Windows Vista. It has the same audio driver support and same audio system infrastructure as Vista. However, it’s some of the under-the-hood improvements that are more significant for audio production. There are some interesting innovations and optimizations in the Windows kernel, making the OS more scalable for concurrent processing. This makes it attractive for highly multithreaded applications like SONAR. Additionally there are various new API’s/SDK’s that may be of significance to developers. Some highlights are below:

Multi-threading: Removal of the kernel “global dispatcher lock”

In Vista and earlier, on a highly multi-threaded system (e.g. SONAR running on an 8 core hyper-threaded Intel Core I7 PC), you have many threads all processing tiny audio buffers at low latency. All these threads are ultimately waiting on the dispatcher lock when it comes time for them to be managed by the Windows scheduler. This global lock becomes a bottleneck in the system and prevents efficient multi-core workload distribution and scalability. This problem gets magnified as you increase the number of cores since they are all gated by a common lock. In Win 7 the kernel team changed the logic in the Windows scheduler to abolish this global dispatcher lock and use per object locks. This effectively removes this age old bottleneck and allows Win 7 to scale better even under workloads of 256 processors.

This change means a lot to applications like SONAR that rely on multithreaded processing of very small workloads. Initial benchmark results have been promising in this regard. SONAR performs more efficiently at low latency on multi core machines.

Improved Memory Management – PFN database lock

The PFN (page frame number) database lock was used by the memory manager to lock pages of memory in the working set. Like the dispatcher lock above, this would gate memory access from different threads causing resource contention. Work in this was first done in Windows server 2003 SP1 and Windows 7 has now has this optimization as well, improving asynchronous access to memory.

Power Optimization: Core Parking

Windows 7 has a new feature called Core Parking. Core Parking is a power saving optimization that shifts processing load to one or more cores and puts other less busy cores to “sleep”. The objective is to let other cores idle if workload levels allow for it. This optimization had us scratching our heads when we ran a benchmark test on a Quad Core I7 machine. At any point in time, we would notice that some cores were idle in task manager. The reason for this turned out to be Core Parking. Core parking can be useful to save battery life while running projects on laptops.

Better WaveRT Performance

Unlike Windows Vista, Win7 now uses event mode internally. This is good news, since it will help guarantee that HDAudio drivers in Win7 support WaveRT event mode properly. Additionally event mode is now part of WHQL logo certification for driver vendors, so any WAVERT device must support this to get a Win7 compatibility logo.

Ed. note: The plain-English translation here is that WaveRT, Microsoft’s own real-time audio driver facility, now is more likely to work the way you expect. Cockos, makers of REAPER, actually provided the ability to turn off WaveRT Event Mode at the end of last year because of unpredictable results. Windows 7 should resolve these issues.

Other Improvements

Peter: Noel also assembled some other improvements worth noting in Windows 7. They’re subtle, but useful: you may finally be able to avoid installing QuickTime/iTunes just to play some video files, interfaces with audio and MIDI jacks don’t have to show up separately any more, there’s improved FireWire support, usability improvements, and multi-touch on mainstream computers is now nearly here.

Noel:

Additional File Format support

Windows 7 adds native playback support for media in MP4, MOV, 3GP, AVCHD, ADTS, M4A, and WTV multimedia containers. It has native codec’s for H.264, MPEG4-SP, ASP/DivX/Xvid, MJPEG, DV, AAC-LC, LPCM and AAC-HE

Yes you read that right – QuickTime MOV file support is now natively available in Windows 7 so you don’t need to install QuickTime. Another big plus is that this is supported under the X64 version of Windows 7 as well, something you cannot do with Apple’s native QuickTime itself!

All media files using these codec’s should play in Media Player. It appears that these new codec’s are exclusively available to Media Foundation applications and not via other legacy API’s such as DirectShow etc.

File format transcoding

File format transcoding of many popular formats is now built into the Windows 7 shell. I.e. dragging and dropping files onto a device automatically performs the necessary format transcoding if the format is supported. This was primarily done to copy formats to portable devices like cameras but should be useful in other scenarios as well.

Multi-function devices and Device Containers:

Prior to Windows 7, every device attached to the system was treated as a single functional “end-point”. While appropriate for single-function devices (such as an audio interface), this does elegantly represent multi-function devices such as a combination audio/MIDI interface. In Windows 7, the drivers and status information for multi-function device can be grouped together as a single "Device Container", which is then presented to the user in the new "Devices and Printers" Control Panel as a single unit.

http://www.microsoft.com/whdc/Device/DeviceExperience/ContainerIDs.mspx

Note: this should not be confused with device aggregation as is available with Core Audio on Mac OS. On the Mac, you can treat multiple audio interfaces as though they’re one interface, so, for instance, you could get extra outputs by combining a couple of audio interfaces, and your software will see them as if they’re just one box. But SONAR provides this capability on its own, so if you’re a SONAR user, you can get the same functionality.

FireWire/USB

Windows 7 contains a new FireWire (IEEE 1394) stack that fully supports IEEE 1394b with S800, S1600 and S3200 data rates. According to reports, USB 3.0 may be supported in a future Windows Update. It was initially planned for Win7 but is not supported in the shipping version of Win7 due to delays in the USB 3 specification.

Multi-touch

Windows 7 includes integrated support for multi-touch displays.

Libraries

Libraries are user-defined collections of content including folders. It’s a handy way to categorize and create shortcuts to samples, music, etc. Special shell folders (Documents, Pictures, Music, and so on) are now Libraries.

Accelerators for Windows

Windows 7 Accelerators provide a way for learning more about selected text, optionally using voice control.

Virtual hard disks

The Enterprise and Ultimate editions of Windows 7 incorporate support for the Virtual Hard Disk (VHD) file format. VHD files can be mounted as drives, created, and booted from.
An installation of Windows 7 can be booted and run from a VHD drive, even on non-virtual hardware, thereby providing a new way to multi boot Windows.

Leaner Footprint

Win7 has a leaner footprint and has been tweaked to work well on less powerful PC’s, laptops and Netbooks. I have heard reports of Win7 working more smoothly on machines that would be slow under Vista.

Listen Mode

Another nice touch in Win 7 is that they now have a listen tab in the audio properties. Turning on "listen mode" basically routes input to the default output device allowing you to monitor an input device in Windows itself. Sadly this runs via the Windows audio engine which is always running in WASAPI shared mode, so it’s subject to a 30 msec delay. Of course you can always load an application like SONAR and route the audio inputs to an output for low latency monitoring.

Compatibility: What to Watch

Upgrading from Vista

Peter: Relative to Vista, are there any changes that are likely to introduce new compatibility issues with hardware or software?

Noel: With any new OS there is always the potential for compatibility issues. Win7 is built on the Vista foundation and one of its goals was better compatibility. As such most applications that are Vista compliant should work as well or better in Windows 7. UAC in Windows 7 has been improved so this might also help with general compatibility problems with some applications.

We have run into only a couple of compatibility issues in Win7 during the course of our development/testing of SONAR 8.5.

The MMIO API in Win7 (typically used for writing RIFF wave files) has a compatibility issue with the mmioDescend API with LIST ‘WAVE’ chunks. This caused our code that reads audio bundle files to fail and read scrambled audio data. We worked around this problem in 8.5

In WASAPI exclusive mode under Win7, the minimum latency you can achieve is now unfortunately 3ms and the code reports an error if lower. The fact that Vista has no such limitation has been reported to Microsoft. Hopefully its a mistaken fence in their code and this issue is fixed via an update, since it’s a step backwards for low latency in WASAPI mode.

Ed.: That last issue is an interesting one for anyone really pushing the envelope with low latency, so I’ll keep in touch with Noel if there’s any update.

Upgrading from XP

Peter: What hardware and software compatibility issues should users be aware of if they’re thinking of migrating not from Vista but from XP to Windows 7?

Noel: The compatibility issues that typically affect users migrating from XP to Vista/Win7 are:

UAC problems: Many applications and plug-ins are not built to handle the newer security settings in these OS’s. For example, if an application relies on something that requires administrative access it will fail when running as a limited user in Win7. This is a serious issue since in Vista/Win7 even if you are running from an administrator account; programs are launched by default with limited user privileges. Unlike XP, you have to explicitly run as an administrator to use such programs. To be Win7 logo-compatible, all applications need to should support running as a limited user.

Drivers: Although for most practical purposes audio drivers in XP and Windows 7/Vista are similar (you still need to write WDM drivers) there are sometimes quirks in specific drivers may cause problems. Most typical driver issues here are caused by installers that make assumptions about the OS version. In many cases this issue can be solved by the end user by setting the “compatibility mode” to Vista in the file properties for the appropriate driver installer file. (Right click the setup exe file to set its properties)

Ed.: I don’t feel either of these is a deal-killer, as I’ve been living with Vista for some time, but they’re still worth watching out for if upgrading from XP. And it means if you have an older machine that’s still working properly, you’re just likely to leave it on XP and worry about sorting the upgrade on a new box.

Less Nagging?

Peter: We talked when Vista came out about User Account Control and particularly audio-specific tasks that required elevation or different handling of permissions in Vista. I know UAC has been streamlined in W7. Do these changes impact audio apps at all? Are there corresponding under-the-hood changes?

Noel: The UAC changes in Win7 are primarily to allow more customization over the UAC elevation prompting process. There are no changes to the fundamentals of how UAC itself works that I am aware of. The classic problem with audio applications with UAC is when programs or plug-ins write to areas of the registry or file system prohibited from standard user access. Even when you are running as an administrator, by default when you launch a program (or the program itself launches a secondary process) Windows 7 will run that process with standard user privileges. If a program or plug-in attempts to write to an area which it doesn’t have write privileges for, virtualization will kick in. While this may allow the program to work, in general it is bad practice to rely on virtualization, since it can cause many unwanted side effects and behaviors in applications.

There are now four customization settings for UAC:

1. Never notify (least secure). The user is not notified when a program tries to install software or make changes to the computer. The user is not notified when they make changes to Windows settings or when programs try to do so.

2. Only notify me when programs try to make changes to my computer. The user is not notified when a program tries to install software or make changes to the computer. The user is not notified when they make changes to Windows settings. However, the user is notified when programs try to make changes to the computer, including Windows settings.

3. Always notify me. The user is notified when a program tries to install software or make changes to the computer. The user is also notified when they make changes to Windows settings or when programs try to do so.

4. Always notify me and wait for my response (most secure). The user is notified when a program tries to install software or make changes to the computer. The user is also notified when they make changes to Windows settings or when programs try to do so.

Customization and Tuning Advice

Peter: How much customization would you advise people do to their OS? That is, you’ve just installed a build of Windows 7 for working with SONAR on a test machine. Do you run the stock configuration, or start turning off services, disabling disk indexing, etc.?

Noel: Optimization and customization is a topic that can’t be fully discussed in the scope of a brief article. In general you need to optimize a system when you have known bottlenecks. Otherwise you can spend a lot of time tweaking things that have little effect on the end goal. In fact, you may even end up destabilizing a perfectly working system. A stock Win7 machine is not optimized for audio necessarily but it appears MS put some thought into trimming out unwanted startup tasks to cut down on startup time. For example there are now “Triggered start services” in Windows 7, so out of the box you can have fewer services running after a fresh boot. There are probably many background services in a modern DAW that could be suspended if you don’t need them but they should be evaluated on a case by case basis depending on what you use the machine for.

Peter: A lot of users were advising running Vista with Aero off, certainly in the early days. Do you think it’s now advisable to leave Desktop Window Compositing switched on for audio work? (Note: I am aware that there’s actually no way to *completely* disable the Aero windowing environment in a way that it reverts to XP, as even in Class mode with no compositing settings the engine has been altered.)

Generally speaking, turning off Aero will free up some resources on your system, since it uses more costly 3D graphics rendering and transparency a lot. However on any modern graphics card, Aero offloads a lot to the GPU so unless your DAW is also competing for the same GPU resources, turning it off may or may not make an appreciable difference to performance. Most applications that are not graphics intensive use GDI for rendering to the screen and since GDI doesn’t take advantage of DirectX hardware acceleration it’s normally not contesting with the GPU. If you are using plug-ins that use Direct 2D or Direct3D, you are probably better off disabling Aero.

Launch Party, After All?

Thanks, Noel. So, the big news behind all of this is that a move from XP to Windows 7 is finally advisable.

I would still caution, as I did recently with Mac OS Snow Leopard, that you typically don’t want to upgrade to a new OS the day it launches. You’ll want to verify compatibility with your software and hardware before making the jump.

That said, this is an unusual upgrade in that it appears to resolve more issues than it introduces. I actually haven’t been able to find a single user out there testing Windows 7 who has found any issues with audio or music production. Of course, when it launches, we’ll have a much larger test base, so I expect we’ll find something – even Windows Service Packs and point releases of Mac OS have been known to create some issues. As we get closer to launch, I’ll review how you would backup your existing XP or Vista system to ensure that if you do choose to upgrade, you can revert to a previous version.

I am, however, cautiously optimistic. And now is an especially good time to make the jump to 64-bit. It’s easier on Windows than any other OS at the moment, and easiest in SONAR, because SONAR allows you to easily migrate 32-bit plug-ins into the 64-bit environment. You’ll need a 64-bit machine and enough memory to make 64-bit worthwhile, but if you’re building a new workstation, as Noel is, the timing could be perfect.

I also think there’s plenty of room left to talk about issues that go between operating systems, particularly how audio software can better support multi-threading and processing on the GPU, multi-touch, as well as emerging I/O standards like USB3. (OpenCL, much-touted in Snow Leopard, is also supported on Linux and Windows, and Linux actually beat both Mac OS and Windows to the punch in providing a first implementation of USB3.) Correction: I should also add that the excellent Reaper has also added this feature. With full 64-bit support in Cakewalk’s own Dimension and other instruments, NI’s Kontakt sampler, and the bundled 64-bit-native plug-ins in Reaper and SONAR, that means you can build a really capable 64-bit rig on Windows.

With fixes getting the OS out of your way, we can return to issues that really matter, many of which apply to every OS.

Music is, as always, the perfect place to talk about these issues. We push our machines harder than just about anyone, and in ways that are the least tolerant of timing discrepencies and glitches. I’ve said it before, and I’ll say it again: if you want to look into the future of computing, ask a musician.

And that calls for a party.

Previous coverage:

SONAR 8.5 and how it can smooth the transition to 64-bit (8.5 is the build that includes Windows 7-specific improvements)

Vista Tweak: Use the Audio Profile Cakewalk’s CTO Uses

Optimizing for Vista: Inside the Mechanics of SONAR 8 with Cakewalk Engineering

Adieu, XP; How Vista SP1 is Doing, and Why This OS Generation Has Been So Tough

Vista for Audio, 1 Year Later: Talking OS Plumbing with Cakewalk’s CTO

Vista for Music + Pro Audio: Exclusive Under the Hood with Cakewalk’s CTO

And yes, I think Noel deserves an Honorary Contributing Editor position for all he’s done giving us absurdly-precise inside details for how Windows works.

Microsoft product screen shot(s) reprinted with permission from Microsoft Corporation.

Code, hardware, and software: Hack Days are all about getting actual stuff made. Berliners, the Music Hack Day that took place in London at The Guardian now gets underway in Berlin this weekend. For anyone who thought the first event was overly Web-centric, there are some new players in Berlin. Ableton is a sponsor, and Ableton, Native Instruments, and RjDj are all presenting hack sessions (in addition to the more Web-focused / consumer-focused 7digital, Songkick, Cloudspeakers, Mufin, SoundCloud, and Echonest). The awesome German musician magazine DE:BUG is also in on the action. I also see our friends at Future Audio Workshop (developers of Circle) in the lineup.

Check out the details:
http://berlin.musichackday.org/
Weekend schedule

So, German readers, who’s going? I’d love to have some spies tell us what the discussions are with NI and Ableton.

I’m, as always, interested in how we can get past geography and share work internationally. So if you’re doing a project, be sure to take lots of pictures, screen caps, code pastes, and the like, and we’ll feature your work here on CDM.

Future events are planned for other cities, and I hope CDM will be involved in some of them. Boston will be the first US event, but it’s on a date I can’t make it. Anyone have a space here in New York you’d like to suggest?

Rounding up my catch-up-on-iPod/iPhone-stories, here’s one from the developer perspective – one that could face music creation developers on the entire platform.

The Apple iTunes App Store now faces the risk of becoming a victim of its own success. Music applications could be a big part of that, without some adjustments on Apple’s part. The problem is this: incoming music “fan” apps could flood out the music production apps that had enriched the mobile software platform since its debut. I think the need could be really urgent. Consider that part of the appeal of Apple’s mobile platform – yes, even in stark contrast to the Google Android on which I’ve been developing myself – is its spectacular real-time audio tools. Combine that with a disproportionately large number of Mac-using musicians, lots of ingenious apps build on Apple’s Core Audio platform, and we’ve seen a mobile platform with an extraordinary number of tools for music creation.

The problem now is that that unique set of powerful apps could get overwhelmed by essentially unrelated “music” apps. A developer who has asked to remain anonymous is already campaigning for a change. He does a good job of explaining the issue, and what might need to happen to fix it. If you’re a developer, you can add your support and feedback to the idea.

Here’s the full explanation:

Hi Peter,
I am writing you with this email I have been sending other developers on the Apple App Store. Although you probably cannot go visit the apple developer connection website because you dont have a membership, I thought you should be informed about the discussion brewing there. Here it is:

Recently the Music section of the App store has gotten very busy with a new type of app, lets call them Artist Apps or Fan Apps. Some of these Apps are great resources for fans and artists reaching out to their fans and potential fans, and some don’t live up to their potential.

However, they are joining a category that previously moved a lot slower, as the apps that had been populating this category were apps with a lot of development put into them and therefore sold at a higher tier usually- but were released at a slower pace. A look at the top 100 paid music apps illustrates this nicely.

Customers perusing the music section to catch that next great sound generating tool (for example), could check in on the new releases section perhaps once a week or even once a month and have the opportunity to check out all the great new apps that had been released, without worrying that one was missed.

Now however, these newer Artist apps have flooded this category, and great apps are getting lost in the shuffle. On one day last week, there were 21 pages of Artist or Fan apps, with a few “other” apps strewn in the mix here and there, very hard to pick out of the jumble.

I understand that this may be happening in other categories for other reasons, but I only concentrate on the Music section since I am a music producer and music App writer.

I propose that we all get together to come up with some suggested sub-category names for the music category. I will start the list off and hopefully some of you will chime in and give suggestions for other categories or add more definition to a sub-category that is alredy here.

Once enough input is received, I will compile it into one bug report for Apple. i will then post the bug# for everyone to include with any correspondance with Apple on this issue.

New sub-categories for the Music section of the App store.

Music Creation:
Synthesizers, drum machines, sound generators, scoring and notation, sequencers, DJ apps, recorders (multi track)

Music Utilities:
Lyrics apps, iPod interfaces, visualizers, iPod controllers, song recognizers, concert finders,

Learning:
Metronomes, guitar and voice tuners, music slow downers, guitar tutors, chord apps,

Artist Apps/Fan Apps:
iLike apps, Deadmou5 app, PVD App, Underworld App, NIN, etc.

Radio Tuners:

AOL Radio, Pandora, Last.fm, individual radio stations

Please visit the Apple iPhone developer forums and voice your opinion/support!

https://devforums.apple.com/message/107989#107989

Definitely voice your thoughts to Apple, but I’d love to hear what you think here on CDM, too (especially since I know some folks at Apple do read this site).

Well, fine, Jamie Lidell. Now you go and ruin it for the rest of us. See, none of us playing with a Nintendo DSi will possibly look as good as you do.

I jest, of course. Jamie Lidell, the wildly-talented vocalist, picks up the new, online-savvy take of the Nintendo DS and breathes cool into it. This is what Sony ads tried to do, but Jamie does masterfully. And, okay, don’t expect the built-in sound app on the DSi to do as much as it appears to be doing here – there’s quite a lot of non-real-time, non-DSi remixing going on, even though what he does do with the simple app is genius.

Thanks to Liz Revision for finding this one.

This does bring us to a burning question: Nintendo and Sony, I’m looking at you. When will we be able to run eccentric and niche music creation apps as official software on your machine? Imagine NitroTracker on the DSi download store or PSPSEQ and PSPRhythm on the Sony Store.

It’s not lost on me, either, that I’ve complained about Apple’s App Store approval for iPhone and iPod touch as being vague and inconsistent with a few apps, while Nintendo and Sony and Microsoft’s Xbox only allow a few apps to ever see distribution. But now that the game makers have online distribution, I wonder if that could change. Movie multiplexes once promised that, amidst a few dozen screens, one would show Bollywood and experimental film. Perhaps these stores could have an “enter at your own risk” category for homebrew. Sony, after all, is desperate to recover sales lost on its PSP. And I have to admit, I think homebrew is, sadly, partly at fault. Homebrew developers and users painstakingly document hacking steps because it’s the only way to get their software on the device – only to have the same system abused by people who don’t want to pay for games.

Korg came out with its DS-10 app for the Nintendo handheld officially, and it was an enormous, runaway success, spawning YouTube virals and entire bands. Now, granted, the app had a major commercial publisher behind it (AQ Interactive), adding credibility – but distribution was limited by the physical cartridge, and the app itself didn’t shy away from Big Boy, niche soft synth controls.

8-bit musicians will likely never touch the DSi, preferring the vintage Game Boy. But a few hard-core gamers are also hard-core handheld musicians. Underground is great, and there’s a certain ethos around hacking. But access isn’t such a bad thing. If just one advocate at Sony or Nintendo would consider it, I think wonderful things could happen.

Now, if you’ll excuse me, I have to see if it’s possible to get a DS homebrew music app running in an emulator on my Android.

“Augmented Reality” is a fancy term for describing ways of using computer vision to overlay digital intelligence on images. In other words, you can, for instance, scratch a vinyl record using a camera – plus a tag for identifying the object’s position in 3D space.

Cambridge-based designer Todd Vanderlin put together an elegant demonstration of the possibilities here, and his video has accordingly been making the rounds. (See: Synthtopia – and I actually heard about it this morning from a high school friend. The power of the Internet.)

Todd has more details on his site, which includes all kind of wonderful projects, like laser sound fountains and, always favorite around here, creepy circuit-bent baby dolls.

AR Scratching [Todd Vanderlin]

There’s actually some work to this: you need to figure out how the album is spinning. And of course, because this is augmented reality and not reality, there’s real potential here to imagine a new kind of vinyl DJing in which normal physics don’t apply.

From the video description:

I was playing around with some AR markers the other day and came up with this idea. taking just a plain old vinyl record and attaching an AR marker to the label you can track the record in 3D space. The next question was, can you scratch the record?

So by figuring out the velocity of the records rotation and applying it to the payback of the audio you can scratch. There is some digital noise that needs to bee worked out, but sounds pretty good. Its still really hard to scratch, it takes some practice but is super fun. The next step is to figure out some nice triggers for different modes. I like the idea of not needing a turntable but the actual spinning of the record helps with the scratching and playback. I made a couple modes, one where the record is paused and you can just scratch through the song. The other looks for zero velocity for x time and then continues on with the song. If there is velocity you then are scratching and the audio is affected. I think that this project has some legs can’t wait to play more.

I Want My Augmented Reality TV

So, this has sufficiently inspired you and you want more augmented reality? We’ve got more for you.

Digital artist and magician Marco Tempest has just demonstrated what happens when you do card tricks with augmented reality – and he shares some details of his rig:

Virtual Magic: Augmented Reality Card Tricks with Marco, OpenFrameWorks [Create Digital Motion]

We’ve even seen augmented reality climbing walls.

If you’re ready to do this yourself, we have a number of resources:

CDMotion has Andy Best’s tutorial on OpenCV with Processing (not an AR-specific library, but relevant): Getting Started, Popping Bubbles

We’ve also got an in-progress library for use with mapping projections in space for “spatial augmented reality”

A New Year’s video with a library for Flash, Java

A tutorial on getting started with augmented reality using Flash

openFrameworks, a library for C++ coding (which in turn supports multitouch, augmented reality – you’ll see some projects on that page)

Bryan Chung is working on a library for Processing

Take a powerful game engine (for animation, 2D and 3D graphics, physics, and on-screen interaction). Add the flexibility of a visual development environment for programming with virtual patch cords, for rich sonic and musical capabilities plus easy interaction with data and input. That’s the idea of combining something like Unity 3D with Max/MSP. In the example from earlier today, the solution simply routed basic data from a Unity-based game to a responsive music engine in Max.

In the case of [myu] – the Max Unity Interoperability Toolkit – that integration goes further still. Developed at the DISIS (Digital Interactive Sound & Intermedia Studio) at Virginia Tech, [myu] allows bi-directional integration of the Unity engine with Max or Pd. The two tools use netsend/netreceive to send data via TCP and glue the two together.

For visualists using Jitter, you can even exchange texture data, which offers some mind-blowing powers for live visuals.

Download at Virginia Tech — bonus, an extension of the aka.wiiremote object so you can use the lovely Wii Fit controller, among various other projects
Discussion on the Unity Community Forums
Discussion on the Cycling ‘74 forum
Virginia Tech DISIS

As an interactive prototyping tool, this should have a lot of potential for lovers of patch-style programming.

Thanks to Dr. Ivica Ico Bukvic, DISIS Director and researcher, for sending in his project. I’ll be curious to see what other people might do with this.