Mythmere the author of Swords & Wizardry has a great series of blog posts on the commercial side of the OSR. The first two are here and here. I think he is doing a great thing by writing these articles and that community will benefit as a whole from his insight.
I have my own take on the situation.
There are two parallel economies in the OSR. There is a commerical economy where a company or author puts up a work for an amount and people buy it. Then there is the gift economy made possible by the extreme low cost of internet and digital access. In the former, money is the medium of exchange. In the latter reputation and access to limited resources, mostly people's time, is the medium of exchange. This is part of a larger shift that our economy is making and has not yet completed.
Of the two the Gift Economy dominates the OSR due to the legacy of Dragonsfoot and other individual communities. One big issues, as Mythmere points out, is that of quality. As the biggest resource that is being competed for is people's time. Time to play, discuss, write, draw, and so on. And people want to spend time on activities involving material that is "good". Of course what is "good" differs a lot between individual but luckily for us, there is the common ground of we want good classic old school material. It what defines the OSR over any other roleplaying community.
In a gift economy people rely on trusted sources to provide an initial sort of what is good and crap. These trusted sources got that way generally because they earned it. There is a distinct first mover advantage but that is mitigated by the fact that one's position can only be retained in a gift economy by being reliable. It some project or individual quality drops it is easy enough to switch elsewhere.
Now the gift economy could have allow me to release the Points of Light for free and a lot of people would have been happy to download it. But it would not allow me to see it in print or see it in stores that have to pay rent and utility for their physical space.
It may be that someday the equivalent of Star Trek's Material Fabricators allows the gift economy to dominate the physical production of items but it is not now. So if you want to do more than see a PDF floating around the internet, you need to become commerical. You build in a profit into your price so you can have the capital to do what you want on the next project.
But going commerical doesn't mean you can ignore the gift economy either. You can if you want but that means you are throwing yourself in the same boat as any other RPG startup company and forced to build your own audience. The industry is littered with the broken dreams of many who tried this. But to be fair many succeeded as well.
If you want to take advantage of the OSR then you need to keep the gift economy side very much in mind. That means not only adhering to the letter of the Open Game License but it's spirit. Charging a reasonable price for your material. Release the occasional free article or product. Particpate in the community through blogging, or the forums. And finally be reliable and honest so that you are viewed as a trusted source.
Finally there is the profit motive. Money is a part of our lives and while it is fashionable in some segments of society to disdain it, I believe that to be driven to make a profit is a good thing in a individual. Provided one is straightforward and honest in their dealings. I have no problem paying for good work for a person who deals fairly, and works hard. And I think if I do good work and deal fairly that people will buy the material I create.
A Setting Book That Would Hit a Few Buttons
1 hour ago