Composr Supplementary: Choosing a software company for your project

Written by Chris Graham

This tutorial explains how to choose a developer.

It's really important to select a company that does much more than just tell you what they know you expect to hear as an outside buyer.

Computer programmers are extremely skilled employees who have to go through degree-level education before they can even truly start the process of becoming experts in the dozens of technologies they work with on a day-to-day basis. It is analogous to solicitors needing to go through law school and doctors having to go through medical school.

This tutorial has a bias towards fixed-bid (fixed price) projects. Ideally you should actually go with an agile project, where you have an ongoing commitment to paying for the project to be tuned and maintained on an ongoing basis. In this scenario there wouldn't usually be a separate maintenance contract or contingency budget, and much of the fine-tuning and testing could be carried on over a longer term. Agile takes into account that no amount of planning can ever be enough, given that lessons are learnt and situations change. However also of course to some it is not feasible to be able to have a project without a fixed budget and guaranteed specification, hence why many clients won't accept a true agile approach. See the Minimum Viable Products tutorial for more details on agile processes.

Terminology

There are many terms you can use to describe a developer of websites (either companies or individuals), such as:

creative agency
IT supplier
software company
engineer
software engineer
developer
web developer
website developer
development agency
programmer
coder

These are all largely interchangeable if it is already implied that we're talking about web development. Even I will not use them consistently. Programming is broader than web development, and engineering is broader than programming. A web developer is an example of a programmer/engineer with specialist knowledge – while of course any random engineer may not even know a thing about programming, and any random programmer may not know a thing about web development.

You may also hear terms like "web designer". A designer typically is closer to the graphic design world, i.e. an artist. In modern teams designers and developers (and copywriters, etc) work together. Nobody can do everything and it's crucial to understand this – but a company likely can. Some companies are more design-born and some are more engineering-born and some are more marketing-born, but generally as mature companies they all grow to be equivalent.

How to evaluate a software company

At ocProducts we often find that prospective clients don't know how to evaluate a software company. It's understandable: a lot of web businesses are first-time-businesses, and they are often started by people who are expert in the subject matter, but not web technology. People can't know everything.

It becomes a problem in two particular scenarios:

when people think shopping for a software company is like shopping for a loaf of bread: find the cheapest thing that looks like it fits the bill (okay, I'm aware some of you like your fancy breads, full of garlic and strawberries, but I'm going to ignore you lot)
when people decide based on the fancy sales pitch that appeared to hit their needs and possibly also came with a low price. This can be very dangerous: it is very easy for a company to invest in sales rather than design or technology, and make strong claims that are actually true but are tied only to very superficial offerings. It is cheaper to sell well than to design and innovate well, and it leads to a wildly successful business model of dealing on volume with a false air of quality that undercuts the actual professionals in the market.

As you can imagine, these scenarios are particular bug-bears for any development company focused around serious engineering – hence why I feel the need to explain how to properly evaluate a software company.

The rest of this section consists of a number of points we believe you should challenge your web company on, and how to test them. To make your evaluation fair, don't tell the company you're reading this tutorial or tell them the expected touch points from your queries, because that would spoil the test.

Of course, not all projects are the same, so use some judgement to determine what particular points affect you.

Evaluating the Requirements Analysis Process

A provider should be able to describe the formal design documents that would be delivered prior to development. Whether these will be delivered as a paid phase, or as part of the bid phase, varies.

You will need to make sure important details aren't missed. Imagine if you were buying a house – you couldn't be confident that you'll get what you want without having a professional discussion beyond your original list of rooms. Never underestimating the importance of planning: failing to plan is planning to fail.

A good developer may create acceptance tests, deployment plans, risk assessments, and UML diagrams. What is appropriate depends a lot on the project and preferred methodology, but you should expect to see at least some kind of professional analysis to be going on.

Evaluating Design

Most people want a unique design for your website, rather than just a template. That's sensible, but don't be completely closed to the idea of a template: you can make a large saving if you start with a template that matches your brand quite well and then modify it enough for it to be unique and tuned to your requirements. You need to raise your level of questioning to much more than "template or not".

We suggest that you ask for the qualifications, experience, and name of the person who will be doing the designing. You should expect them to be a professional designer (not a programmer or jack-of-all-trades) and for them to have many years of experience or a proper design school education. You should also ask to see designs they have specifically done themselves (don't expect them to necessarily be done for the web design company you are talking to, it is perfectly reasonable for designers to move between companies or freelance).

Ask about how they will make the design reflect your brand principles. You should expect a response that shows a good understanding of brand theory.

Evaluating CMS Programming

Ask what CMS will be used (maybe you want Composr CMS, but this tutorial is non-partisan), and what experience they have writing custom code for that CMS or very similar systems.

If they just mention a third party CMS and say they use third party plugins for it, and your project isn't just a very off-the-shelf kind of thing, run for the hills. You need someone who can actually make things, not just plug them together.

Be aware that some companies will deploy "their own CMS". In my companies case this means Composr, which is an enormous CMS we have built and release as Open Source. However, in the vast majority of cases it will mean either:

They have a really simple system that only does a very small number of things and is unlikely to be maintained well
They use someone else's CMS and call it their own (sometimes its just word-play, sometimes its using a white-label system)

Don't let a developer get away with this kind of sneakiness.

You should ask to see examples of specific new functionality the developer has implemented themselves for their chosen CMS.

Evaluating Security

Ask what particular process is used to ensure new code is secure. Ask them on the phone (i.e. without giving them to prepare) what a "CSRF vulnerability" is. Of course, your phone contact may be a manager, but you can ask them to connect you to a developer. If they cannot connect you to a developer, the "we don't actually do programming in-house" flag should raise.

CSRF stands for "cross site request forgery" and is a vulnerability where a hacker creates an 'evil' third party website and persuades an administrator to go to it, and that 'evil' website redirects a request over to the administrators website to instruct it to do something on it like delete something. If they cannot explain this clearly, you should be extremely concerned that the programmer is not experienced (there are a lot of programmers who have a very focused/limited knowledge of things). Don't let them come crawling back later about how they have "learnt new things", because this will just be illustrative of hundreds of critical holes in their knowledge and a lack of competency and professional integrity: CSRF is just one of dozens of areas of important security consideration a developer should know of.

Ask them to describe the process for how a second senior developer reviews code security. They may say that the primary developer is very senior, which is okay if that really is true.

Evaluating Past Programming Experience

You should not expect the company to be able to show another project they have done which is very similar to your own, as this is unreasonable. However, you should look to see projects of a similar kind of complexity and seek to ensure they understand the general traits of your website. For example, expect to see experience writing social websites if you need social functionality, or at least get them to show they have attained insight via other means.

And, of course, you should ask for the credentials of who is doing the programming. At least find out how many years of experience they have and whether they have an engineering degree – or whether they have a really strong portfolio you can't argue with.

Evaluating eCommerce

If you are doing credit card processing, ask them what you need on the server for this to work, and any official processes you need to go through. They may say you should use an external processor, like PayPal, which is fine, but ask them to explain what you would need to do if it was all on-site. They should mention a PCI compliance audit, and buying an SSL certificate. A manager might not know this, but between a manager and a developer (you may need to talk to both), somebody should.

Evaluating Project management

Ask what process will be used for managing the project. Ideally they should be able to send you a diagram, because a good web development company will have something in place already as a 'default' process. Of course, every project is different, so they may tune it for you, but you should expect more than just words.

The process should clearly show at what point you may ask for revisions, when things are 'signed off' and it should reference charging models for scope changes and how this may impact schedule.

If this has not all been thought through then you really have to question the experience and competency of the company.

Evaluating the Maintenance Contract

You should expect an ongoing relationship with the web developer, as you can't realistically launch a website and not have any updates made to it. The web is constantly advancing, and new web browsers come out all the time which should be compatibility-tested.

You should explain you require ongoing service from your web developer, and ask them to propose how they will charge for testing of new browsers. Mention you need to ensure that the site stays secure and ask them what they suggest – they really should mention to you a way of them rolling out security updates to you if vulnerabilities are found in the software. Expect to pay for it, but expect it to be offered too!

Evaluating Price

The pricing tutorial provides some detail on what are typical prices to expect.

If a provider is quoting significantly less than expected for the kind of company than they are, try and assess whether they can sustain their pricing. They might have underestimated complexity, be charging less than they are worth due to naivety, just trying to make some money on a side-gig, or just trying to win your work so they can do a quick and dirty job on it for you.

On the other hand, they may have some particular process or approach, or personal drive, that makes them able to compete particularly well.
Do a probing analysis rather than just comparing quotes.

Evaluating other factors

Here are 11 more things to consider:

If the main developer/manager is ill or on holiday, is someone automatically always there to pick up the project? Is there some kind of audit trail of work performed, or at least access to shared notes?
What source code management system is used?
What issue tracking system is used?
What is the policy for out-of-hours support? (weekends, evenings, the middle of the night)
Is there a documented off-site backup policy already in place, including regular testing of backup integrity?
Are all developers based in (home country), and if not, quantify the cost saving this will give me?
What contingency budget should the customer retain for unexpected development costs?
How much should the customer set aside for marketing the project, outside of development?
Is the developer working full-time for you, or do they have other commitments and distractions?
Is the developer committed for the long term and able to sustain their operation?
Does the developer have professional or personal contacts with people working for Silicon Valley companies, or other top IT companies? The best developers will regularly network with people, rather than living in a local bubble. The networking isn't necessarily anything to do with career building: it is a natural by-product of being at the top of your industry that you will befriend and work with others who also are. This is especially true given the prevalent use of Twitter by developers.

Evaluating yourself

I can't stress enough how important it is for you to consider a business model for your website.

You need to perform a market review and come up with some kind of plan on how you will differentiate yourselves against your competitors, how you can beat them without them just copying your innovations, and how you will reach your future customers. If your differentiation is weak, you also need to consider how you will be able to create a superior solution than your competitors on the budget you have.

Remember that you are not competing with your competitors as they are now, you are competing with them in the future when the project is finished (it takes time remember, and things can happen in that time). Not only this, you need to consider the situation where all you end up doing is raising the bar in the industry when your competitors just absorb your own ideas in their next updates.

Plan defensively and strategically! And, budget accordingly.

And…

Budgets are always limited, so to fit your budget a web developer probably will not suggest all the things that would be a good idea for you. Can you blame them, they don't want to hear a thump as you fall onto the floor in shock? Besides, it takes time for them to make suggestions, and it is futile for a web developer to suggest what they think you can't afford. It's pretty common for web developers to have future clients coming in for a project that costs ten times more than the client thinks even in its most basic implementation, because web pricing really isn't that well understood (see my pricing tutorial).

Here are a few things I would not necessarily expect a web developer to suggest, but if you can provide a large enough budget are quite possibly worth having included:

Production of a high quality introduction video for your front page
A print stylesheet, so your pages look good when printed
A favicon (that's the little icon for your website that shows in the address bar and bookmarks)
Testing across a whole range of different smartphone and tablet sizes
Usability testing (highly recommended, but potentially relatively costly)
A staging site, so that you can test and experiment with your CMS
Ongoing study of analytics to find weaknesses, and propose improvements
Ongoing Internet Marketing support, such as social media campaigning and sending of nicely designed newsletters
Ongoing SEO for link building and tuning position on some applicable search terms
Accessibility testing with a screen reader, and automatic testing tools, to ensure blind users can use the website effectively
A full code review by a senior engineer
A proper training course for any site administrators to go through

A larger set of possible work items is included when contacting ocProducts.

Offshoring

A very large proportion of web development on Western websites is secretly done in Eastern countries. It's the elephant in the room, or in this case, the Indian Elephant in the room.

There's a huge advantage to offshoring PHP development: cost. The numbers seem extraordinary, $15 per hour vs $100 per hour.

In this tutorial I will try and explain when offshoring PHP development works, and when it does not.

First, let's discuss a bit of context. When a company says their rate is xxx per hour, you need to consider what that actually means. Comparing to a serious Open Source engineering company like my company ocProducts:

We work on an Open Source model, so the customer is essentially paying for the maintenance of the ecosystem they are working in within the hourly rate, but as a result there is a bit of a monopoly of expertise: you're paying the people who created the ecosystem, so there'll be a huge efficiency from that.
In our case, often there is a certain lead of project management time that doesn't get counted in the hours. A project will be spec'd up, and we do try and charge for it in project management, but it is rarely equal to the amount actually spent setting up a job. When offshoring you typically do absolutely everything on the clock, because there's no lower service tier that the offshoring companies need to differentiate themselves against.
When we use our hourly rate, it is almost always on a fixed-quote basis. The reality is we usually spend 2 or 3 times longer than we actually account for, because the hours estimated are usually quite optimistic, and in reality there are always things that come up, like bugs in third party code to workaround.

So after all that, when comparing like to like, we are probably talking about more like $15 per hour versus $20 per hour (assuming our efficiency saving made us get 50% more done per hour).

That's a big drop, but it's still cheaper to offshore, and when you run a business you do need to be sensitive to costs, because if you aren't, your competitors will. So the happy-smiley feeling of paying Open Source developers can't be a deciding factor, unless of course you have a requirement to push the Open Source project forward because nobody else is willing to do so and you need that particular project in your business (Composr is healthy enough regardless).

People in India (or Pakistan, or Russia, or any other country) are not inherently any more or less smart than people in, say, America or the UK. To say that is pure xenophobia and I find it highly distasteful.

So, where can offshoring go wrong? Here's a list from my own experience:

An issue called power difference comes into play. Developing countries usually have a culture where it is considered highly unprofessional to ever question someone who is a superior. Managers in Western countries usually thrive on feedback from people under them, and ask for all their decisions to be questioned, and I think this is vital to success. You really don't want people to be "yes men". Don't underestimate the seriousness of this problem, a manager never has the time and perspective needed to perfectly explain a task, but if employees don't ask questions when they start to explore out the task in its full detail, those tasks don't get done right. This kind of culture is inherent to countries where labour is very cheap, because it's more about power structures than cost.
Time-zone differences: either you need to wait for a developer to come on-shift, or the developer is asked to work horrible hours and then sleep in a noisy city during day time (Indian cities are notoriously noisy).
You can't look over someone's shoulder to see how people are doing. A manager has to find a balance between high-level management and micromanagement (the balance is different for different employees), and looking over people's shoulders occasionally is a great technique that is completely impossible with staff who are not in the office. Of course, this problem also applies to freelancers and people working from home. Don't think that Skype or desktop sharing will make up for this, it is far harder to get an accurate check using these things, and it takes a certain amount of negotiation and time each time you want to do it. I can say from experience that just walking around the office (maybe while walking to the door to go to the loo – i.e. taking out virtually no time for it) and checking everyone at once is infinitely better. Never assume people will pro-actively bring problems to you, because developers are all to at least some small extent 'shy', they try not to bother you, and they often do not realise how they could be working more efficiently.
People are less likely to ask questions if they have to type them out.
I have good Indian friends who know English far better than me, but the average Indian developer will make a lot of grammatical mistakes that need to all be reported or fixed in-house.
There are differences in quality standards between developed and developing countries. This may sound controversial, but it cannot be denied, ask any Indian and they will say how their country is organised chaos. People in the West have grown up with an expectation of beautiful refined media, not things that are hammered together, and you often see this difference in the quality of online work too. It sounds prejudiced, and perhaps it is, but I think it is true for many cases (of course, you should never generalise and stereotype, it's a thin line to walk!).
Offshore developers don't feel compelled to compete so hard to justify their rates, because there is nobody significantly cheaper for you to run to. Of course, many people have better reasons to compete than just based on rate justification, but I think to some degree the basic feeling of "you're paying peanuts, just how hard should I bust my ass for you" is sometimes there. Again, I don't want to generalise, because I have seen some extremely diligent Indian developers.
The demand for offshoring PHP development exceeds the supply of developers, yet the price ceiling cannot rise due to the nature of the market, hence breaking normal rules of supply and demand – hence kind of tying up the industry in a certain mediocrity.
The more experienced developers usually either leave the country, or they get tied up in large corporates that work for companies like Microsoft, or they take themselves out of the talent pool by starting their own businesses.
You cannot doubt that there will be cultural-based and language-based communication problems. To take a common example, pink in India is often a favourite colour for men, which is actually the natural case because it is a dangerous blood-like colour which goes hand in hand with your stereotypical male aggression – but a few hundred years ago that changed in Western cultures and it became female-associated.
It is very hard to explain some things in writing, compared to demonstrating something on screen and using your finger to point.
Most offshoring companies will not let you interview developers yet also don't have the same level of written programming standards a Western company would have.

As you can see, the situation is far more complex than just the "communication issues" cliché, and it is not an issue with core skill.

But even after all this, you cannot deny the difference in price, it still often works out cheaper when you take everything into account. Consider these counter-arguments:

It is cheaper per-hour regardless of the above factors
The great Western developer you hire may not be available to you either. I mentioned $100/hr above, but you could find that the developers you really need charge $200/hr because the $100/hr are fully booked or don't have the decade of practical experience you need. There is a significant shortage of skilled software developers and salaries have gotten very high.
To retain developers, you need to maintain a relationship where you give them regular work. Even if there are efficiency problems with offshore developers, being able to keep them permanently on salary is a huge advantage.
Having people in different timezones can be useful. You can pick up work in your country, after it was finished in another – or get things done conveniently over-night so they are waiting for you in the morning.
You may be able to find a really good offshore developer who doesn't have many of the common problematic traits above, especially if you pay them over-the-odds. There are stand-out geniuses in every country.

I have a lot of personal experience offshoring to India. For most of the life-time of my company I have kept at least some offshore developers on-staff, and continue to do so. Always work has had to be heavily scrutinised before releasing it to clients, but having a balance of onshore and offshore skills has always worked well for me, even though it is often very stressful to manage it. My opinion is that you use offshoring PHP development if you have a task that meets the following criteria:

It doesn't need much explanation
It is quite long
It involves standard skills
It is not particularly cultural

I hope you find all this useful, and balanced. I have considered these issues for years, and it's hard to get past the stereotypes, clichés, and biased points of view.

Feedback

Please rate this tutorial:

Have a suggestion? Report an issue on the tracker.