Software-as-a-service (SaaS) is one of the most successful business models today. In 2021, every company subscribed to about 110 SaaS applications on average. In many companies, SaaS solutions have become essential components of their critical business processes. So, acquiring visibility into their SaaS applications is no longer optional but a crucial part of managing their enterprise risks.
SaaS monitoring concerns itself with injecting that visibility into SaaS operations. In this article, we start with an overview of monitoring and SaaS monitoring. We then explain some key security areas that a good SaaS monitoring solution should address. Finally, we explain the benefits of doing that and some best practices that can help you as a SaaS customer (or a provider, to some extent).
What Is Monitoring?
In general, monitoring involves collecting, processing, and visualizing (using dashboards) the metrics of different aspects of your software systems. It provides real-time full visibility into their operations and helps achieve your operational and performance goals.
Depending on the aspect or goal, a few of the types of monitoring are:
- Infrastructure monitoring: This monitoring measures low-level aspects like system uptime, CPU usage, memory usage, storage usage, system load, GPU load, network latencies, and more.
- Performance monitoring: This measures metrics related to performance at every level of your stack, from the infrastructure at the bottom to the end-user applications at the top, covering everything in-between — operating systems, platforms, and libraries. It enables full-stack end-to-end diagnosis and optimization of performance issues.
- Application monitoring: This measures metrics specific to the unique concepts and software semantics of a particular application.
- Application performance monitoring (APM): This measures the performance of the features provided by an application. It’s a subset of performance monitoring but its focus is on a single application’s components and not on end-to-end diagnosis.
- Experience monitoring: This measures metrics that reflect the perceptions of end-users and their user experiences rather than metrics reported by software or hardware.
What Is SaaS Monitoring?
SaaS monitoring is collecting the metrics of SaaS applications for operational, performance, and security goals. SaaS monitoring is an important component of SaaS management. As such, a SaaS app is just another type of application, and you may think SaaS monitoring is no different from normal application monitoring in your data center.
However, the deployment model of a SaaS makes its monitoring somewhat different from on-premises application monitoring. A SaaS has two distinct use cases:
- SaaS customers who use a SaaS
- SaaS providers who provide a SaaS
Both should implement SaaS monitoring on their end of the relationship. But their goals and techniques are quite different.
For service providers, SaaS monitoring is just like any other on-premises web app monitoring. SaaS performance is measured by instrumenting it just like any other app.
But for customers, SaaS monitoring involves a different set of techniques because the application is not deployed on their premises. We’ll explore these differences in the rest of this article.
What Is SaaS Monitoring From a Security Perspective?
From a security perspective, good SaaS monitoring should also collect the metrics that are relevant to the company’s cybersecurity.
Some metrics are not directly security-related but prove helpful in diagnosing security incidents. For example, an unusual increase in network activity to a SaaS endpoint may be an indicator that your system has been compromised for a botnet or denial of service attack.
Other metrics may be directly related to security, like counting the security events collected by the logging infrastructure. For example, an increase in failed authentications may indicate a brute-force attack.
We’ll explain what makes for good SaaS monitoring from a security perspective next.
13 Areas of SaaS Monitoring for Security
By understanding SaaS interactions and use cases in-depth, we’ve come up with a list of 13 key target areas for your SaaS monitoring. These security-focused monitoring goals are primarily meant for SaaS customers. SaaS customers like you should look for SaaS monitoring tools that can process the monitoring data and derive the metrics recommended by these goals.
However, they’re not just for SaaS customers. Each goal guides SaaS providers too in how they should design and offer their services. A responsible provider should strive to support their customers in each area by providing sufficient visibility into their services.
SaaS providers can do so by publishing security metrics and logs through application programming interfaces (APIs) or webhooks. A webhook is a customer-owned HTTPS URL interface where a SaaS provider can send supplementary information.
1. Identity and Authentication Metrics
Identity and authentication infrastructure consists of multiple components implementing concepts like identity providers, single sign-on (SSO), security assertion markup language (SAML), OAuth 2.0, OpenID Connect, multi-factor authentication servers, and federation.
Each of these components should collect the metrics that can indicate cyberattacks, like:
- Sources and rates of failed authentications by type
- Number of failed credentials grouped by type, IP address, and other useful attributes
SaaS providers should publish the related metrics for each customer through an API.
2. Authorization and Access Control Metrics
An authorization is a permission granted to perform some action on a resource, like permission to edit a document. Access control is the runtime evaluation of various security attributes at a particular point in time to decide if an authorized person is allowed to access that resource. The evaluation is guided by your company’s access control policies.
In a SaaS environment, access control is the SaaS provider’s responsibility, and most customers leave it to the provider. But some customers may have stricter policies that are enforced inside their perimeter even before any request reaches the provider. Components like proxy servers and cloud access security brokers (CASB) are used for such internal enforcement.
Every step — configuring access control policies, granting authorizations, and access control decisions — can be targeted by different types of cyberattacks. All software components implementing these steps should collect metrics that are useful for cyberattack detection and forensics, like:
- Number of access control policy and authorization changes in a period
- Details of who’s making the changes — IP addresses, identities, departments, and so on
- Number of successful and failed access control decisions in a period
Every SaaS provider that implements access control should publish the metrics of decisions through an API.
3. Secrets and Credentials Metrics
An employee may sometimes need to sign in to a SaaS using a privileged account like a root or administrator account. They request the credentials (like passwords) from your secrets management solution. The latter should log the identity, time, and other request details with your logging solution.
Your monitoring solution should collect the metrics of these events, grouped by request and time. An unusual ramp-up in credential requests may indicate a compromised administrator account or secrets server.
4. User Action Metrics
User actions are the things your employees are doing in a SaaS web application to accomplish their assigned tasks. From a security perspective, the idea is that most employees on most days follow the same routines. So their actions in a SaaS also follow some set patterns.
If the monitoring solution notices anomalies in these patterns, like the odd timings of actions, it may indicate an insider threat or a compromised account.
But detecting these actions on the customer side is not easy. They are specific to each SaaS and occur inside a browser session, a black box for all practical purposes. That’s why SaaS providers must publish user action metrics grouped by user and time.
5. Configuration Change Metrics
Every SaaS supports some configuration settings over the entities it manages. For example, Google Drive enables an owner to decide if the editor role can grant permissions to other roles. Administrators and users may change SaaS configurations to make their tasks easier or for troubleshooting some problem.
Whatever the reason, configuration changes can quietly expand your attack surface, opening up opportunities for cyber attacks. That’s why these changes must be visible to security teams. A good monitoring solution should collect configuration change metrics like:
- Who is making the changes and how often
- How many objects are potentially impacted
A configuration change is a type of user action but with potentially deeper consequences. Monitoring them has the same complications for customers as the other user actions described earlier. SaaS providers must publish configuration change metrics grouped by user and time.
6. API Metrics
Many SaaS applications offer their functionalities through APIs to enable their use in mobile, desktop, or command-line applications or to support automation by SaaS customers. The number and type of API requests going out from your network can point to possible cyberattacks — like malicious insiders, data exfiltration, and compromised credentials.
7. Webhooks Metrics
Webhooks are an alternate option for SaaS providers to provide important supplementary information, including security metrics and logs. Webhooks are especially helpful for long-running tasks and continuous reporting. Stripe is a good example of a SaaS that sends a wide variety of payment information through webhooks.
Your monitoring solution should measure metrics like the number of webhook requests by URL. Your HTTP servers should implement ways to publish the supplementary information they receive — especially metrics and logs — to your monitoring solution.
8. Network Metrics
All monitoring solutions collect network metrics — like bandwidth and latency — to support operational and performance goals.
They can help your security goals too. Network metrics and statistics are useful indicators of compromise. Unusually high network traffic from your network to a SaaS may indicate cyberattacks like a denial of service or data exfiltration. You should combine monitoring solutions and automated security workflows for remediation, including automatically updating firewall rules.
9. Infrastructure and System Metrics
Other commonly collected information like infrastructure metrics and system metrics may also contain useful indicators of compromise. For example, high CPU or GPU usage can indicate crypto mining malware in your systems. If they are seen in the same systems where a particular SaaS client library is installed, it may indicate a vulnerability in that library.
10. Threat Management Metrics
Threat management and threat scanning tools look in logging and monitoring data for the indicators of compromise reported by global threat intelligence databases. Some of these indicators may involve metrics and logs of SaaS applications. The information the tools need should be collected by your monitoring and logging infrastructure. Plus, the tools should report back detection and remediation metrics to your monitoring solution.
Frequent threats involving a particular SaaS or its library indicate poor security practices of the SaaS provider. It may be time for you to look for a new provider.
11. Vulnerability Scan Metrics
SaaS usage involves many aspects like web application requests, API requests, client-side libraries, their dependencies, domain lookups, security certificate verification, and more. Each of these may contain some vulnerabilities.
Vulnerability scans check if your systems match the reported versions and environments. That may allow attackers to exploit those vulnerabilities and impact either your or the SaaS provider’s systems.
The vulnerability scanning tool should report its metrics and findings back to your monitoring and logging systems. Frequent vulnerabilities involving a particular SaaS or its library indicate poor security practices of the SaaS provider.
12. SaaS Alerts and Notifications
Many SaaS providers offer alerts and notifications for various events. A common one — usually offered by SaaS applications with pay-per-use pricing plans — is alerting you if your aggregate billing amount for the month crosses some thresholds. An unexpectedly high bill amount may be due to a non-malicious error like bad software logic but may also indicate a malicious reason like compromised credentials or crypto-mining malware.
13. Service Level Indicators
SaaS service level indicators are useful for both your operational goals and security goals. Measuring KPIs (key performance indicators) like SaaS uptime and downtime may reveal a SaaS that is not reliable. Frequent outages, slow response times, and high MTTR (mean time to resolve) may hint at their poor operational or security practices. Often, a SaaS that’s bad at operations is probably bad at security too and may pose a risk to your data security.
4 Benefits of SaaS Monitoring
As a SaaS customer, is it worth investing your money and time into a capable SaaS monitoring infrastructure that’s security-conscious? Here are four benefits of doing so.
1. Supports Your Business, Operational, and Security Goals
The business intention behind any monitoring is to serve your customers and partners with the best possible availability, reliability, security, and speed. Monitoring and logging give you real-time insights and prior warnings in all these aspects, enabling you to preemptively avoid bad consequences.
2. Manages Your SaaS Security Risks
Your company may be just a SaaS customer, but that relationship still brings security risks from cyber attackers, bots, SaaS providers, partners, contractors, employees, and others. SaaS monitoring helps detect, prevent, and sometimes even remedy these risks.
3. Supports Your Service Level Agreements
Service level agreements (SLA) are based on measured service level indicators and thresholds. SaaS monitoring helps you track and verify that your provider’s SaaS service levels do meet those agreements. It’s not just about legal obligations. A SaaS provider that fails to stick to their SLA can reduce your service levels to your customers and damage your reputation.
4. Helps With Regulatory Compliance
Regulations like the Health Insurance Portability and Accountability Act (HIPAA) expect security for sensitive health data, including the data you store with your SaaS providers. Your company may have to comply with other national, international, or industry standards.
SaaS monitoring provides objective metrics that can inform your compliance and risk department how your systems are faring.
3 Best Practices for SaaS Monitoring
Here are three best practices we recommend for your SaaS monitoring infrastructure.
1. Collect and Report Comprehensive Metrics for Each SaaS
This applies to both you, the SaaS customer, and your SaaS providers. Monitor the 13 target areas we listed above. Whenever possible, select the SaaS applications that provide in-depth visibility into their functionality, operational status, and security events to your IT teams. Without such measures, you’ll be driving blind, probably straight into a disastrous security attack or loss-making downtime.
.svg)
