Assure Service Description
Outpost24 Appsec Assure is a point in time security assessment of an application designed to provide customers with the ability to provide a third party or internal audit team an assurance of the security level of the application, that focuses on vulnerabilities and issues that would be deemed to be critical or high. Outpost24 delivers this level of assurance to organizations, using a combination of highly experienced ethical hackers, tools and the Appsec portal.
Each Appsec Assure engagement is designed to be short-term, delivering the following services:
- Assurance focused test of the application
- Unlimited 24/7 support
- Zero false positives
- Multi-factor authentication testing
- Context-aware risk scoring
- Ability to verify individual findings for 30 days
- Secure web portal for presentation of results
- Ability to raise comments and questions on findings through the portal
- Ability to export Appsec Assure testing report from the portal.
The Appsec Assure service is a single engagement per Web application. All findings, once verified for false positive removal, are presented to the organization through Outpost24’s Appsec Portal.
For the purposes of the service, A web application is defined as:
"A collection of web resources composing a client interface, and the exposed server-side services used to orchestrate this, perceived by a user as one entity".
One web application is not necessarily limited to one domain, subdomain or URL and may be hosted on multiple servers.
Instances are counted as separate applications for the purpose of the Appsec Assure service.
1 Access to the applications
The Appsec Assure analyst team is based in Sweden and Denmark. The testing will originate from the following network range:
To avoid interference, this network range should be whitelisted in the network and web application firewalls (WAF) protecting the web apps if there are any. Interference may otherwise lead to reduced testing coverage, depending on the conditions.
The network traffic may be sent from multiple different machines having different IP addresses in that network range and therefore there is no single IP address to whitelist. Outpost24 owns the entire 18.104.22.168/24 so there shouldn't be any concerns about traffic coming from anyone else than from Outpost24.
2 Application scoping
When a customer is ready to commence an Appsec Assure engagement, a scoping request should be submitted to Outpost24. Organizations should submit the required application information using the Outpost24 Appsec portal, alternatively a submission via email to the Snapshot@outpost24.com email address can be made using the Appsec Assure Engagement request form.
All applications to be assessed must be externally facing web applications. Customers may submit multiple requests per month, however final start dates of the requested engagements is subject to Analyst availability. It is recommended customers provide at least five (5) working days’ notice before commencement of the testing is required.
3 Test accounts
At least one user account is required to test authentication, authorization, access restriction, and other related multi-tenant functionality.
To achieve full test coverage, the following should be provided:
- At least one user account for applications featuring authentication
- At least one user account per unique user role or user level (e.g. one administrator and one regular user)
User accounts may be omitted if a self-service sign-up exists (e.g. a public account registration).
We recommend using the secure credential service in the Appsec portal to send credentials. However these can also be included in the scoping submission or via an encrypted email as required.
4 Applicable documentation, tokens and hardware
Some applications may have specific requirements regarding certain operations or functions. Common examples include:
- Order operations, which require a credit card number
- Multi-factor authentication, which require a hardware or software token
- APIs, which require detailed documentation to achieve adequate testing coverage
When completing the scoping request, the customer should provide the means required to execute all regular functionality. The Appsec Assure Analyst will raise an inquiry whenever a test case cannot be run because of an unmet requirement. At such point the customer can opt to either provide access or exclude the functionality from testing.
5 Explicit exceptions excluded functionality and test cases
It is possible to exclude testing of certain areas, functionality, or test cases, if desired. In such case, a full exhaustive list of what is to be excluded must be included when submitting the scoping document.
The following tests are not executed as a part of the Appsec Assure delivery:
- Denial of Service (DoS) attacks – no DoS attacks will be performed because they may cause the web application or server to cease normal operation during the test.
6 Engagement duration
During the engagement period, the in-scope web application is tested, the results verified and uploaded to the Appsec portal for the customer to perform remediation activities. As needed, the customer may ask for a verification of a finding, post comments or questions on a finding to seek clarification or answers
The testing of the application will be conducted over a period of three (3) working days, with all verified findings posted to the portal, and an executive summary added within seven (7) working days from the commencement of the engagement. The customer may then continue to request validation of findings for up to a further thirty (30) days before the application assurance test will be marked as completed. At the end of the thirty (30) day period the executive summary will be updated to reflect all remediation activities concluded and the final status of the application.
7 Scope changes
Once agreed, the scope of the engagement is fixed for the duration of the Appsec Assure engagement. Additional scope changes will be subject to the use of an additional Appsec Assure engagement.
The engagement process used for each penetration test of a web application consists of the following steps:
1 Testing performed
To see a list of the areas tested under the Assure service please review the information on the Outpost24.com website at the following location.
2 Vulnerability feeds
Appsec Assure uses Outpost24’s own vulnerability database built with data gathered from multiple publicly available vulnerability databases such as NVD, OSVDB, vendor advisories and the Outpost24 research team and is updated at least daily.
3 Impact and production safety
Appsec Assure was designed live, public web applications and therefore causes less impact than traditional approaches. Since all the tools are operated by trained analysts, the actions of the tools are strictly controlled and monitored to prevent them from being disruptive or affect the state of the web application. The vulnerabilities are reported as soon as they are confirmed, and no further exploitation is performed unless the customer gives us permission to proceed.
To clarify, consider these examples:
The scanner contains a crawler module. The analysts will refrain from invasive testing but should there be content on a page which causes an unexpected behavior (i.e. non-circular error logs,) there may be situations where an error log of a production environment component fills the storage units. This may cause a permanent DoS effect on the system level. This is nothing the analysts can observe as an external tester/assessor, but it will be observed as the system going down. This would happen to the system eventually with or without the analysts’ involvement as the logs grow, but it is triggered by the testing.
If XML External Entities (XXE) are tested, this can be done via test cases for an echoed response, a remote request/inclusion, or via an XML entity expansion. The Appsec Assure analysts as well as the Appsec Assure scanning platform will never perform the last of these tests since it will likely crash the web server engine on the system by causing an out of memory exception. The other methods are non-disruptive and will be used.
DoS vulnerabilities are not in scope of the Appsec Assure security testing, any potential DoS vulnerabilities are coordinated with the customer before any additional testing occurs.
The amount of traffic generated by Appsec Assure is as high as needed to crawl the web application, test the input parameters and gather data used for change detection. This traffic is also tuned to decrease the impact on the performance of the web application.
The amount of the generated traffic depends on the size of the web application and therefore no exact measures can be provided. To prevent unnecessary load on the web application, the following approach is applied:
- De-duplication of requests leading to the same resources
- Downloading of static resources like documents and media files is limited
- HTTP requests may have thresholds applied or delayed in multiple ways
The impact on the web application can usually be compared to a few users visiting the website and performing some actions in the web application simultaneously.
As a result, Appsec Assure is intended for production environments, but its production safety is context dependent. It is far less intrusive in its test cases than fully automated web application scanners which will issue potentially disruptive SQL statements as attempted injections.
Appsec Assure Deliverables
1 Verification of findings
To eliminate false positives, all findings are verified and confirmed by Appsec Assure analysts. During the verification process the analysts clearly describes the recreation steps and include these in both the portal and the Assurance report to simplify remediation efforts by the Development teams.
2 Appsec Portal
Unlike many other pen test services, Outpost24 believes organizations should be presented with findings in an online portal for easier tracking, follow up, assignment and remediation. As such, customers will be granted access to the Appsec portal where they may review all findings, and generate assurance reports as required
A customer will be limited to a total of three (3) accounts unless agreed ahead of time with Outpost24.
Within the portal customers can perform several functions. These are
- Ask questions or comment on a finding, customers can comment or ask questions about each finding presented for that application. When doing so, an Appsec Assure Analyst will respond to the comment or question, as appropriate, within one (1) business day.
- Request validation of a finding, customers may also select one or more findings in the portal and request re-validation. Once requested the Appsec Assure Analyst will endeavor to retest each requested finding. This is aimed at those organizations who are passing findings to development teams to remediate. Customers may request as many findings to be re-validated as they wish, however the service will only guarantee a maximum of 10 requests to be validated each business day. R-evalidation of findings are available to the customer for up to thirty (30) days after the test has been completed. Optionally customers may purchase an additional verification service to increase this by a further thirty (30) days. Contact your sales account manager for more information.
- Generate an Assurance Report, customers may use the portal to generate an Adhoc Assurance report. As findings are remediated, the report will reflect the most up to date state of the application.
Appendix A – Appsec Assure service levels
99.8 % availability measured on a monthly basis
|To be agreed within five (5) business days of receipt of the request.|
|Verification of finding by Appsec Assure Analyst within one (1) Business day of receiving the verification request from the customer with a maximum of ten (10) verification requests within one (1) single day|
|Response from Appsec Assure analyst within one (1) business Day of receiving the comments from the Customer|
Appendix B – Engagement request form
Whilst its preferred that submissions for engagement requests are made directly through the Appsec portal. Should you wish to email in a request (firstname.lastname@example.org) Please complete the following Appsec Assure Engagement template).
Preferred Start Date
*List any specific administration URL’s
Administration Interface URLs
*Any specific part of the environment that is out of scope as described by the customer.
Out of Scope URLs
*Any sensitive functionality that was described by the customer and that may be affected by security testing.
Description of Known Sensitive Functionality
*Usernames and passwords listed by the customer to be used during the tests.
Credentials and Role Description