Composr Supplementary: How to approach complex projects

Written by Chris Graham

Composr is designed as a tool to lower the investment (be it time or money) needed for making advanced websites. Typically it might reduce that investment by around 50%, but it varies greatly from one project to another.

Complex projects

Something I see quite regularly in the Composr community is people "going it alone" on pretty complicated IT projects that go beyond standard features and assumptions.

Often people don't realise their website project is actually quite a complex software development project.

To be frank, most of the time it is probably a mistake to go it alone for something more than a personal site. DIY is admirable, ambition is admirable, initiative is admirable, and Composr is awesome-powerful – but building IT systems has never been easy, and content management systems aren't a magic bullet that can allow arbitrary complexity with just clicks and plumbing code. Even professional developers struggle to make efficient and user-friendly systems.

We'll get back to DIY users later, don't despair if you are one – there is a lot of space for you within the Composr community.

The value proposition of Composr is that it will do a large proportion of you want virtually for free. But, very rarely 100%. You can have forums, chatrooms, databases, newsletters, a wiki, member accounts, permissions, all set up with clicks, so that's enormous value to you. But, creating arbitrary complex and customised user-friendly websites on only available (pre-existing) functionality is less feasible.

Expect to be able to create a general-purpose social network with a tonne of features easily. Don't expect for complex projects to be able to replace the need for database designers, systems analysts, programmers, and graphic designers (and possibly project managers, copywriters, and SEO professionals).

An example

Let's imagine you want to build a custom site which is going to tell the user comprehensive information about wildlife surveys on the island of Jersey. You have "animals" and "sightings", both current and historic, with over 100k records, and you have cross-linkage.

The approach many people have is "let's build a catalogue for animals, and a catalogue for sightings, and customise the templating and add a search form".

However, the reality is you'll going to come across things you haven't thought of, and you've already started your project with an assumption of "this is easy, and I am going to do it myself". You might not even have thought much about that basic assumption, but it's a dangerous one, and a bad precedent to start a project on.

The truth is, there is a lot to know about database design. There are all kinds of theoretical concepts, and practical requirements of how to optimise querying by using indexes and optimising queries. Catalogues are built on the simple idea of viewable and content manageable entries on a website, and there's basic support for field relationships, but that's a long way from a full database system, and it's a long way from a professional database designer properly optimising things.

Trust me, any of the successful complex database systems you've seen on existing websites were built by a professional team.

Consider this quandary:
You've built a basic catalogue, and imported 100k records into it, and you decide you need a more customised editing interface than a sequential list of field inputs. At this point you've worked hard to get going, but you've something that has a rather complex implementation (catalogues are pretty complicated internally in order to support the flexibility and automation), with a rather simplistic user interface. Even if you decide to hire a programmer to finish things off, it's going to be hard for them as the basic underlying structure will not be efficient for custom coding. The programmer may insist on throwing out current achievements and starting again with something more appropriate.
(I know I've just talked about catalogues being flexible, then said a situation where they're not. You have a tonne of options to control how a catalogue is built, but full control of a user-input form is always going to have to be a coding-level task.)

What Open Source really means: how commerce and community connect

Let's take a step back and focus on DIY users again, because they are important, nay, vital. They are a minority when it comes to implementing complex projects, but they are also the life-blood of Composr.

In our opening paragraph we talked about your website being an investment. In many cases the investment users make in their website is just in time – our users spending time learning the Composr software, and taking advantage of the great features it has to offer. These are the DIY users, suited to basic personal sites, or advanced sites when these users are very skilled and/or willing to learn and invest a lot of time.

This way DIY users get a better website than if they had used a lesser tool, or save tens of thousands of pounds/dollars/euros that you'd need to pay for a ground-up development from a traditional web development company. Really passionate Composr users go further, and maybe learn XHTML, CSS, or even PHP – and from this, push Composr as far as they want. The most advanced users are often the people who contribute directly to the Composr product (via the Open Source process) and/or come on to become web professionals.

Sometimes (often for businesses), the investment might be in money rather than time, and go to a web development company (such as ocProducts). Businesses and organisations can pay professional developers to make a whole website using Composr.

In-between there's a proportion of users who want a balance – to invest their own time, and also to pay for some professional services. This usually means users will make websites themselves in Composr, and pay a developer to add some extra features, or to provide some direct technical support for the software.

As a company, ocProducts makes it's money from the services we provide to users wanting professional services. We wrote 99% of Composr and Composr is free, so we don't make anything at all from that. We choose to give it out for free because we love the whole concept of it – what it empowers people to achieve (especially the DIYers) – we are able to give it out for free as it provides a foundation for the services we provide.
A consequence for this is that when we are providing services to people we are acting as a business, earning money to pay our salaries – not just directly for the work we are doing, but also to fund what we do for free. We are able to do this and still charge less because, as Composr experts, we are able to save those many tens of thousands of pounds/dollars/euros for our clients.

This way of operating is called an Open Source business-model. Other companies do this, such as Zend (the maker's of PHP), Oracle (the maker's of MySQL and Open Office) and Redhat (one of the major sponsors of Linux).
Open Source business-models are innovative. They allow companies to give out enormous value in software to communities, completely for free. We love that we can do this, even though in most cases we don't earn anything back.

In summary, the Composr community involves DIYers, as well as businesses purchasing services. Additionally, ocProducts operates within the community and also commercially. The commercial and community worlds connect together, for everybody's mutual benefit. Sometimes it is messy and awkward, but the benefits are enormous.

Further on in this tutorial we will provide some guidelines for you to identify whether you are a DIYer, or need to be thinking about purchasing professional services. This will be determinate based on your skill, time, quality requirements, complexity requirements, as well as your ability to find a budget.

Pricing

Pricing is mainly covered in the Project pricing tutorial, but we will touch on it here also from a more philosophical point of view.

Often people who are in communities aren't accustomed to the kind of figures that companies have to pay for things, and expect services to be cheaper, more like the price you see for things in shops, or pay for a software licence. There's a big gap between 'free'/$100 and 'tens of thousands of pounds/dollars/euros'. The reality though is most websites for businesses cost this much, or often a lot more because they are developed by a permanent staff. There are ways to get savings – you can outsource it all to developing countries, but unless properly managed there is almost always a big quality gap – and often it will still cost a lot.
Prices are relatively high, because wages are – think how many $100 there are in the yearly salary of someone who has been through years of university training, and then add to that things like the cost of dead time (periods where incoming work is slow), administration costs, write-downs (e.g. non-paying-clients), training, marketing and PR, management, sickness leave, vacation periods, tax, expansion funds, contingency, office and bills, expected return on investment, and many other costs. For people outside business costing, the actual cost things come to is a bit of a shock, but the figures that go around are fair reflections on the costs of it all.

Open Source web developers often get a situation where people come looking to purchase some services, but don't have past experience of business costs. Most people don't. This creates a problem for, when potential clients have an unrealistic expectation of costings – and think that a quote for a few thousand for a website, of one thousand for a new module, is extortion – when in reality, it would cost 10 times more if they approached a traditional web development company. By explaining the background to how costs are calculated, and what we can do for free, I hope we can demonstrate how we maintain an enormous passion for Composr and Open Source, while showing that our professional services are necessarily aimed at organisations with a realistic business budget, and not intended to be taken up by the majority of our users (community websites).

A good approach

So, what I would advise is at the start of any complicated project, really question how you're going to approach it. Here are some key questions you should have answers to:

How popular is this expected to be?
Am I willing to learn programming myself?
Am I over-complicating things, is there a simpler way?

Your answer to the first question effectively should define financing. When you start a complex project, you should be thinking about the return on investment, even if it's a non-profit project. Is the emotional return worth it? How much will it cost in money or time, compared to how much people are going to benefit from the project?

If the return on investment seems poor, you should really be thinking about how you could just do something simpler.
If the return on investment is good, you should be thinking about getting an investor(s). It might not be money, it might be recruiting passionate volunteers who share your goals. In this scenario you are being a leader and manager, rather than a lone-wolf. The lone-wolf model doesn't scale, especially nowadays on the mature web.

My over-arching point is, starting with the assumption of going it alone on a complex IT project, is probably not a good idea. You're just setting yourself up for pain. You may well be digging yourself into a deep hole without realising it.

The second question is pretty obvious. If you want to do something complex, do you want to learn programming, or hire a programmer? As described above, you'll probably need a programmer for at least part of it. Learning programming could be a great thing, it'll improve your career prospects, it's fun, it's rewarding – but there's no denying that it's hard and time-consuming and takes a particular kind of mind.

And third, it may well be things can be vastly simplified. Going back to my wildlife survey example, do we even need a database front-end on the website? Why not just use the downloads addon to have a spreadsheet or Microsoft Access database on the site? Why not just link to a Google Docs spreadsheet? Maybe you can link to a full screen HTML file that contains a table? Maybe there's even a pre-existing tool that can export a Microsoft Access database to a network of hyperlinked HTML files out there. Sometimes things may be a lot simpler if you just try and reign things in to a much simpler and more standard use-case. It comes down to, does the example really need 100k fully content managed entries each supporting comments and ratings, or is the really requirement just to get data out there?

So, based on your thought, you probably will reach one of these conclusions:

Project is quite straight-forward, I really can do it alone (great!)
Project is complex, but actually it doesn't need to be, and demand is very niche anyway – I can vastly simplify
Project is complex, and demand is high – I want to learn programming
Project is complex, and demand is high – I want to hire a programmer
Project is complex, but fortunately aligns perfectly with standard features, the limitations and assumptions and architecture are not being pushed (really check this is true, but if it is, great!)

Then, before you start the project, I highly recommend writing a proper set of documents for the project. This may include a business plan, a financial plan, a sitemap, wireframes of each screen, user role analysis, documented user stories & journeys, database structure, and so on. It is much better to get things nailed down in writing rather than diving straight into it. Good up-front analysis can help you avoid many mistakes further along, and can be shown to other people to review and get feedback.

In conclusion

I hope I haven't stamped on too many people's dreams. I want to re-iterate that Composr is the best CMS out there for massively reducing cost. It's just very commonly people use that as rope for hanging themselves because they incorrectly assume that "reduced" means "all but eliminated". I want to save people needless pain for all our users, who we love.

Feedback

Please rate this tutorial:

Have a suggestion? Report an issue on the tracker.