Technology Audit for Web Services

In our blog posts, I often write about things that might come in handy when developing new websites and web applications. However, there are also many interesting and useful topics associated with existing web resources too. Today I’m going to touch on just one of those topics: technology audits of web services.

At SECL Group, we have plenty of experience working on projects that are already in progress. In most cases, our work on further development begins by conducting a technology audit, and so we have a wealth of knowledge to share on the subject. When a project owner approaches us for an audit, they typically want to understand the project’s strengths and weaknesses, and how to best address any identified issues. They also seek guidance on how to proceed with the project in the most effective manner possible.

In this article, you will learn about why technology audits of web services are carried out, and cover the main stages to be aware of when doing so. I will also talk about typical features of the technology audit for web services, examine case studies and share recommendations for conducting a technology audit for web services.

Technology Audit step by step

Before we get into the details, it’s important to understand that the term audit is a broad concept in web development, and that whilst we make reference to the overarching concept of a technology audit in this article (otherwise known as a technical audit), keep in mind that there are many types, which are performed differently depending on the goals that have been set.

Commonly however, we either carry out an overview audit of the entire project, or a thematic audit of specific parameters and/or issues of a website or web application.

As an example, we recently completed an audit of the loading speed of a web project, following which we were able to make recommendations for improvements to the customer. According to forecasts, these changes should speed up loading times on the project by up to 4-5 times.

The documents provided by the audit team will vary depending on the purpose of the technology audit. However, a report containing recommendations for improvements should always be included, although the structure and focus of the report may differ. Typically, the more clearly defined the audit objective is, the more favorable the outcome will be. Therefore, as a golden rule, we advise that you be transparent with your audit team and share all of your concerns to achieve the best possible result.

Step 1. Preparation of technology audit of web services

If the goal/s of a technology audit of web services has been defined, then the preparatory stage can begin.

1. First of all, both the customer and auditors should discuss whether or not granting access to the production version of the project will be necessary. Sometimes, a test version, or even simply the code in the Git repository are provided for audit. This is often the case in financial services projects, where security issues are very acute and managers do not want to allow outside specialists onto the production server. Where auditors have not been asked to check download speeds, then this condition can be accepted, but where download speed is to be examined, then it’s better to check it on the production server as it will provide accurate readings.
2. Next, auditors will need documentation, but too often this is either outdated or even missing. Whilst not an insurmountable obstacle, insufficient documentation will inevitably lengthen the time the audit takes, so best practice is to request that the previous development team either generate or revise the documentation. We have detailed the various types of documentation in a separate article, but at the very least auditors require an explanation of the project’s architecture and technology stack, deployment instructions, and API documentation, if applicable. Additional available documentation can also prove to be useful.
3. A description of any ‘technical debt’ would also be helpful. This term refers to any accumulated problems or necessary improvements that have been shelved and not yet implemented. ‘Technical debt’ is usually described by the developers who wrote the project, but they often do this reluctantly since it pertains to their own shortcomings.
4. It is desirable to provide the audit team the opportunity to communicate with the project’s developers. This is not always possible, especially when it comes to the potential transfer of the project from an old team to a new one, which we have also written about in another article, but where such communication can be achieved, it will certainly help reduce the time spent on the audit since the auditors will receive prompt answers to their questions.

Step 2. Conducting a technological audit of a web project

In a technology audit for web services, the auditors’ own expertise is critical. Top specialists are usually involved, which as a rule means either seniors, or most often, team leads. Auditors are not only looking to reveal problems which have already manifested themselves, but also potential new ones.

Of course, a customer will expect actionable recommendations for improvement based on their audit results. But it is important to be sensible about this. Any programmer will always want to say: ‘We need to throw everything out and rewrite it correctly’, but at SECL Group, given our track record, we understand that when working in the best interests of product owners, auditors should be actively looking for ways to deliver maximum improvement with minimum effort. This approach distinguishes a professional team of auditors from amateurs.

As for deciding upon the composition of the technology audit team for complex projects, it is useful to consider the back-end and front-end separately. Depending on the specifics of the project and the objectives of the audit, it is also possible to involve other specialists in addition to developers.

Now let’s analyze the audit process itself.

First, auditors will unfold the project, look at its basic settings, and analyze its architecture. Sometimes mistakes are found within the architectural solution itself, and such errors significantly affect the entire project. It is undoubtedly a demanding task to change the architecture of a working web project, but it is possible. To help mitigate risks here, we quite often recommend splitting monolithic web applications into microservices, which significantly speeds up the application, reduces the number of errors, and allows new programmers to get up to speed much faster.

In the second stage of a technology audit, code is checked and individual parts of the project are examined. The entirety of a project’s code is usually not studied, since there can be a lot of it, but the overall quality of the code can and should be assessed. It is also valuable in identifying recurring errors and highlighting best practices. During the technology audit for web services, the quality of the development is revealed.

It is of course possible to conduct an audit of individual specific problems and tasks. For instance, you could perform load testing to find weaknesses in application performance, or it could also be useful to identify security weaknesses. But be selective about what you choose to focus on, as overreaching will be long and expensive. Prioritize the objectives of your technology audit and focus on those, remembering that the most problematic aspects of any project will be obvious to those with the appropriate expertise.

In addition to assessing the quality of the architecture and code, the audit covers the following main groups of key web services parameters:

• System Development and Implementation
• System Performance
• Security
• Documentation
• Standards and Procedures.

For each of these sections, auditors prepare a list of issues that should be clarified during the technology audit, and therefore a kind of checklist is compiled. This makes it easier for auditors to record facts and prepare conclusions. Amongst checklist items, experienced auditors will always highlight verified markers that signal which individual parameters of the web service need to be more carefully studied. Systematization of technology audit issues in checklists later greatly simplifies the preparation of an audit report. More on that later.

Sometimes, alongside the audit, it is possible to provide the customer with related services, such as, for example: testing. Such services can be provided both at the separate request of the client or at the initiative of the contractor. To make an additional positive impression on the client, usually when auditing individual parts of a web project, we always look, at least in a general sense, at other parts of it.

If, during such a review, we discover any serious shortcomings or threats, then we inform the client about them. For example, if we see poor code and a lack of unit tests, then we instruct our QA engineer to spend several hours finding bugs in critical parts of the project. By doing this, working in the interests of the client, we are highly likely to find further important errors.

By going beyond the scope of a technology audit of web services, we can more fully and accurately demonstrate to clients the state of affairs in their web projects, as well as confirm the weaknesses and risks discovered during the audit process, and provide specific examples. This comprehensive approach always pays off in practice.

Step 3. Audit Report and Recommendations for Improvement of Web Services

After conducting an audit, your team of specialists produces a report that outlines the findings and suggestions for enhancing the web resource. The project owner should be informed not only about any bugs that have been identified, but also about the most effective methods of addressing them, as well as the anticipated impact of each solution. The experts must also specify the priority order in which problems should be addressed.

Diligent, professional auditors provide clients with information not only about the problems that have already occurred but also about potential risks and threats. This empowers product owners and managers to make informed decisions regarding the operation and future development of their web project.

The layout of the audit report generally follows the plan and checklist as discussed in step two. However, it’s important to note that you’ll only find all of these issues outlined together in a comprehensive audit. The structure of the report is tailored to the specific purpose of the audit, so if only a single problem or project component is being examined, only a portion of these elements would be included. Additionally, this checklist is useful for self-evaluating the status of your web services.

System development and Implementation

Design and development: determination and collection of development needs, design and development procedures, entry data, and consistency of development stages and elements.

Testing: complexity, rigor of tests, correctness of testing.

Implementation: compliance with the procedures and standards for the approval of changes, their implementation, ensuring system security during and after implementation, documenting implementation and post-implementation review.

Web project owners and developers can reap numerous advantages from expert assessments of their software’s performance and adherence to industry standards. These evaluations include verifying the proper functioning of various web resource modules, ensuring high-quality layouts, optimizing graphics, and configuring servers and extensions, among other factors. As a growing number of users now access web products via their personal mobile devices, it’s also essential to evaluate the performance and accuracy of mobile versions of sites and applications in the audit report.

Separately, checking compliance with software development standards is also crucial. At SECL Group, when conducting an audit, depending on the specifics and technologies chosen for the project, we rely in particular on the following well-known international standards:

• General programming guidelines: Coding conventions, MDN Web Docs, Naming convention.
• Python: PEP 8
• JavaScript/TypeScript: ECMA, JavaScript Standard Style, Google TypeScript Style Guide, ESLint.
• PHP: PSR.
• C#: ReSharper.
• HTML/CSS: The recommendations of The World Wide Web Consortium (W3C), etc.

System performance

Outages: Outage frequency, mean time to resolve and time between failures, total system downtime.

Storage and utilization: Utilization of RAM, hard drive storage, and cloud storage. The use of cloud solutions is so significant in modern web development that this topic should always be considered in the audit process. Plus, there’s usually room for improvement suggestions. In cloud services, the amount of resources used depends on the workload, so by optimizing the load, you can avoid unnecessary costs for excess resources.

Network performance: Network latency, upload and download speeds. (We recommend that these parameters be evaluated separately for different parts of the world. From experience, it is known that something may load quickly in the US or Canada yet too slowly in Europe, for example).

Security

Evaluation of program code in a security context: Auditors check to what extent the development complies with security standards and whether the project team have adhered to secure coding practices. It is necessary to determine any security vulnerabilities, which can not only be found in the developed code but also in any solutions taken from external libraries, as well as open source technologies. In the report, the auditors express their opinion on the presence of security vulnerabilities and associated risks.

The audit report also evaluates the levels of protection that safeguard the developed web system from prevalent threats, such as:

• DDoS attacks i.e. distributed denial-of-service attacks, whereby a huge number of incorrect external requests from many IP addresses make a system inaccessible to users. These attacks are high-profile and massively disruptive.
• Cross-site scripting (XSS) attacks that inject client-side scripts into web pages visible to other users. An unfortunate consequence of a system’s vulnerability to XSS attacks can be, for example, redirecting user requests to malicious web pages.
• SQL injection, which is particularly dangerous for data-driven applications, is carried out by entering malicious statements in Structured Query Language (SQL) to access the database or perform actions on the server. The failure of the software to resist such attacks can lead, for instance, to the leakage of sensitive data, etc.

Of course, this list is far from exhaustive and is constantly evolving. Attackers purposefully look for flaws in the software code of web services that would make hacking possible. Therefore, it is essential that auditors discern whether there are errors in the code and if secure programming techniques were used when writing it. In this way, the audit team evaluates the system’s ability to resist malicious attacks, as well as spell out any corresponding risks.

Server firewall: Installing and regularly updating a server firewall, including intrusion detection and prevention systems (IDS / IPS).

Server anti-virus software: Installing, keeping active, and regularly updating antivirus software on all servers. Installing and properly configuring patches immediately after each incident.

Hardware: Compliance of all devices with the minimum hardware requirements for the correct operation of security programs. Password protection for all devices.

Passwords: Compliance of passwords with security requirements. Regular password changes. Blocking of accounts after incorrect logins to the system.

Accounts: Removing deactivated accounts. Account information encryption.

Alerts: 24/7 monitoring of all alerts for unauthorized access, unplanned system modifications, and any system intrusion.

Documentation

Completeness and relevance of web service documentation: Auditors make sure that the documentation does not have significant gaps and covers all elements of the development and operation of the web service. It is also mandatory to check that documentation is up-to-date.

The composition of documentation depends on the features and technologies of the project. As a rule, well-trained project teams compile and routinely update documents on thematic sections, such as:

• Software architecture design. Fundamental solutions for building the system, description of its elements. Interrelations of the system with the environment and data flows.
• User interface (UI) and user experience (UX) design.
• Source code and technology stack. Guidance on coding, if adopted and applied by the development team. Any applied back-end and front-end frameworks, content management systems (CMS) and content management frameworks (CMF), templates, plugins, tools, and other solutions from external libraries, etc. are listed.
• Security rules, if they have been set for the project.
• APIs (Application Programming Interfaces), if used during development.
• Installation and deployment, including integration with third-party solutions.
• Quality assurance, including testing and a list of applicable standards and requirements.
• Maintenance.

Security protocols: Proper documentation, regular updating, and distribution of security protocols. Updating security protocols after each system modification and any security significant event.

IT logs: Protection of IT logs from tampering, regularity and timeliness of their reviewing, and compliance with retention periods.

Incident reports: Accuracy of incident descriptions, their causes, and solutions. Conducting necessary updates to procedures based on the analysis of incidents. Evaluation of the impact of each incident on the system.

Standards and procedures

Backups: Keeping a daily backup of important data. Regularity of verification and confirmation of backups. Having 2+ separate storage locations for backup files, including at least one external location.

Sometimes, at the request of the customer, an audit is not limited to the verification of its technologies and technical parameters, and project owners may seek a comprehensive analysis of their projects. In such cases, the study of the website becomes a business technology audit. Consequently, the necessary specialists with relevant profiles are also involved in the audit team, and the structure of the report is also expanded upon. Other similar extensions in scope may see the inclusion of additional thematic audits, such as:

• SEO audit
• Usability audit
• Mobile-Friendliness audit
• Audit of monetization tools, etc.

Of course, this list may well be extended due to, for example, various aspects of marketing analysis, economic performance, and user experience of the web service.

As demonstrated, skilled auditors can effectively select and utilize methods to examine all aspects of web services through ongoing dialogue with their client. Additionally, it’s worth noting that auditors typically provide recommendations for correcting website or web application deficiencies and collaborate with the client to devise an action plan for implementation.

Conclusion

So there you have it — this article has covered all the ins and outs of planning and conducting technology audits for web services. You can see that audits for web development can be very diverse, and the quality and characteristics of an audit will largely depend on its goals and the know-how of the team carrying it out. It’s crucial to make sure that the audit’s scope and complexity match the size and intricacy of your web project. When it comes to selecting a team for an audit, be sure to go for those that have experience and expertise in similar technologies and projects.

Our area of specialization lies in conducting audits for Python, JS, and PHP web projects. Clients usually come to us for a technology audit of web services when they switch contractors or when they’re dealing with specific problems that need attention. Often, experts can spot a variety of problems and suggest solutions even without carrying out a full audit.
If you have any questions or concerns about your web services, don’t hesitate to reach out to us. We can offer guidance and advice in many cases even without conducting an audit, which can save you time and money. We’ve helped plenty of clients in this way, and we’re confident that fresh perspectives and practical advice from experienced web developers can help you tackle your project’s issues and prevent future ones.