Leaving docker stats
running in a terminal is often useful to see which containers are running and how much work they’re doing.
Adding the --debug
flag to the end of sam local
commands can be helpful.
We have tried these launch configurations. The fact that you are expected to add your environment variables in yet another place, specify your runtime again, and add your event again, along with the fact that some environment variables would override and some wouldn’t was enough to make us stick with our existing VS Code launch configurations.
provisionAppEmissaries
purpleteam-s2-containers/
root directory. Run the following command:
docker run --name docker-compose-ui -v $(pwd):$(pwd) -w $(dirname $(pwd)) -p 5000:5000 --rm --network compose_pt-net -v /var/run/docker.sock:/var/run/docker.sock francescou/docker-compose-ui:1.13.0
purpleteam-lambda/
root directory using sam local start-lambda
. Run the following command:
sam local start-lambda --host 172.25.0.1 --env-vars local/env.json --docker-network compose_pt-net
provisionAppEmissaries
from aws cli. From the purpleteam-lambda/
root directory, start 10 containers:
aws lambda invoke --function-name "provisionAppEmissaries" --endpoint-url "http://172.25.0.1:3001" --no-verify-ssl --payload '{"provisionViaLambdaDto":{"items": [{"testSessionId":"lowPrivUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"lowPrivUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"lowPrivUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"lowPrivUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"lowPrivUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""}]}}' local/app-emissary-provisioner/out.txt
Just be aware on your first run that the stage two images are going to be fetched, so it won’t be instant
docker stats
running, verify that the containers were started with the following command:
docker container ls
purpleteam-s2-containers/app-emissary/
directory:
docker-compose down
deprovisionS2Containers
Lambda as seen belowprovisionSeleniumStandalones
purpleteam-s2-containers/
root directory. Run the following command:
docker run --name docker-compose-ui -v $(pwd):$(pwd) -w $(dirname $(pwd)) -p 5000:5000 --rm --network compose_pt-net -v /var/run/docker.sock:/var/run/docker.sock francescou/docker-compose-ui:1.13.0
purpleteam-lambda/
root directory using sam local start-lambda
. Run the following command:
sam local start-lambda --host 172.25.0.1 --env-vars local/env.json --docker-network compose_pt-net
provisionSeleniumStandalones
from aws cli. From the purpleteam-lambda/
root directory, start 10 containers:
aws lambda invoke --function-name "provisionSeleniumStandalones" --endpoint-url "http://172.25.0.1:3001" --no-verify-ssl --payload '{"provisionViaLambdaDto":{"items": [{"testSessionId":"lowPrivUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"lowPrivUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"lowPrivUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"lowPrivUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"firefox", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"lowPrivUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""},{"testSessionId":"adminUser", "browser":"chrome", "appEmissaryContainerName":"", "seleniumContainerName":""}]}}' local/selenium-standalone-provisioner/out.txt
Just be aware on your first run that the stage two images are going to be fetched, so it won’t be instant
docker stats
running, verify that the containers were started with the following command:
docker container ls
purpleteam-s2-containers/selenium-standalone/
directory:
docker-compose down
deprovisionS2Containers
Lambda as seen belowdeprovisionS2Containers
purpleteam-s2-containers/
root directory. Run the following command:
docker run --name docker-compose-ui -v $(pwd):$(pwd) -w $(dirname $(pwd)) -p 5000:5000 --rm --network compose_pt-net -v /var/run/docker.sock:/var/run/docker.sock francescou/docker-compose-ui:1.13.0
purpleteam-lambda/
root directory using sam local start-lambda
. Run the following command:
sam local start-lambda --host 172.25.0.1 --env-vars local/env.json --docker-network compose_pt-net
deprovisionS2Containers
from aws cli. From the purpleteam-lambda/
root directory, run the following command:
aws lambda invoke --function-name "deprovisionS2Containers" --endpoint-url "http://172.25.0.1:3001" --no-verify-ssl --payload '{"deprovisionViaLambdaDto":{"items": ["app-emissary", "selenium-standalone"]}}' local/s2-deprovisioner/out.txt
docker stats
running, verify that the containers were brought down with the following command:
docker container ls
No local Lambda service needs to be running first, in our case:
sam local invoke
commands:
provisionAppEmissaries
, from the purpleteam-lambda/
root directory, run the following command:
echo '<same-JSON-payload-as-above>' | sam local invoke --event - --env-vars local/env.json --docker-network compose_pt-net provisionAppEmissaries
provisionSeleniumStandalones
, from the purpleteam-lambda/
root directory, run the following command:
echo '<same-JSON-payload-as-above>' | sam local invoke --event - --env-vars local/env.json --docker-network compose_pt-net provisionSeleniumStandalones
provisionAppEmissaries
, from the purpleteam-lambda/
root directory, run the following command:
echo '<same-JSON-payload-as-above>' | sam local invoke --debug-port 5858 --env-vars local/env.json --event - --docker-network compose_pt-net provisionAppEmissaries
provisionSeleniumStandalones
, from the purpleteam-lambda/
root directory, run the following command:
echo '<same-JSON-payload-as-above>' | sam local invoke --debug-port 5858 --env-vars local/env.json --event - --docker-network compose_pt-net provisionSeleniumStandalones
app-emissary-provisioner
In VS Code:
We used a similar launch.json as mentioned here
Open the Lambda handler and put a break point where you want it to stop.
The VS Code instance needs to be restarted after any changes to the launch.json.
sam local start-lambda
command from the purpleteam-lambda/
root directory:
sam local start-lambda --host 172.25.0.1 --env-vars local/env.json --docker-network compose_pt-net --debug-port 5858
aws lambda invoke
command as discussed previously from the purpleteam-lambda/
root directory:
aws lambda invoke --function-name "provisionAppEmissaries" --endpoint-url "http://172.25.0.1:3001" --no-verify-ssl --payload '<same-JSON-payload-as-above>' local/app-emissary-provisioner/out.txt
Just be aware that if the stage two containers have not yet been fetched that it will take some time to do so
app-emissary-provisioner
folder, switch to debug (Run) view, click the “app-emissary-provisioner (purpleteam-lambda)” project in the drop-down, click debug arrowapp-scanner
and sub-processesUsing VS Code, we have created a launch.json to use for this.
Make sure you have the app-scanner project loaded in VS Code.
sam local start-lambda
command as detailed in the Full system runpurpleteam-orchestrator/
root directory:
npm run dc-build-debug-app
npm run dc-up-debug-app
runTestSession
routine). This is useful information for taking through the rest of your session and correlating the app-scanner logsprocess.id
command in the VS Code Debug Console, this will give you a PID, which you can correlate with the PID you captured from the app-scanner log in the previous step, along with the test session Id. From here if you want to debug the other test session, just click on it in the Call Stack pane, and check the process.id
again. This way you can always know which test session you are debugging.Discussed here
From the purpleteam
root directory you can run the following command:
npm run debug
Or if you need to override a NODE_ENV environment variable to say local
then run:
env NODE_ENV="local" npm run debug
Or if you actually want to exercise say the test
command, run the following command:
env NODE_ENV="local" npm run debug -- test
In Chromium, open chrome://inspect
and click “inspect”, which will drop you into the first loaded script.
From the purpleteam
root directory you can run the following command:
npm run test:debug
Or if you need to override a NODE_ENV environment variable to say local
then run:
env NODE_ENV="local" npm run test:debug
In Chromium, open chrome://inspect
and click “inspect”, which will drop you into the first loaded script.
Leaving docker stats
running in a terminal is often useful to see which containers are running and how much work they’re doing.
docker container ls
is also quite useful for watching port allocations.
docker-compose-ui needs to be running. Running it in it’s own terminal is a good idea to see what is happening.
You need to use the user-defined network already created in order for Lambda function in (what was previously docker-lambda) the AWS managed Docker Image to be able to send requests to docker-compose-ui
. From the purpleteam-s2-containers/
root directory, run the following command:
docker run --name docker-compose-ui -v $(pwd):$(pwd) -w $(dirname $(pwd)) -p 5000:5000 --rm --network compose_pt-net -v /var/run/docker.sock:/var/run/docker.sock francescou/docker-compose-ui:1.13.0
Additional resources:
http://localhost:5000/api/v1/projects
will list your projectschrome
service in the purpleteam-s2-containers/selenium-standalone/docker-compose.yml
file:
curl -X PUT http://localhost:5000/api/v1/services --data '{"service":"chrome","project":"selenium-standalone","num":"2"}' -H 'Content-type: application/json'
docker stats
or docker container ls
purpleteam-s2-containers/selenium-standalone
docker-compose.yml
down
Host Lambda functions:
From the purpleteam-lambda/
root directory run the following sam local start-lambda
command. Running it in it’s own terminal is a good idea to see what is happening:
sam local start-lambda --host 172.25.0.1 --env-vars local/env.json --docker-network compose_pt-net
The --host [gateway IP address of compose_pt-net]
is required to bind sam local to the user-defined bridge network compose_pt-net
in order for it to be reachable from the app-scanner container.
The following are the links that were useful for working this out: host.docker.internal
, extra_hosts
and other comments from here down, docker-host container, extra_hosts
reference, along with creating the firewall rule as mentioned above, and testing connectivity as mentioned in the Docker page with shelling into running container
Start your SUT (NodeGoat in this example). Running it in it’s own terminal is a good idea to see what is happening:
There are at least two options as mentioned in the set-up
Run the docker-compose to bring the stage one containers up. Running the following commands for this step in their own terminal is a good idea to see what is happening:
purpleteam-orchestrator/
root directory, run the following command:
npm run dc-build
purpleteam-orchestrator/
root directory, run the following command:
npm run dc-up
orchestrator
with app
):
purpleteam-orchestrator/
root directory, run the following command:
npm run dc-build-debug-orchestrator
purpleteam-orchestrator/
root directory, run the following command:
npm run dc-up-debug-orchestrator
purpleteam-orchestrator
, or purpleteam-app-scanner
process within the container. Further details in the Debugging section
Start cli:
If you are running in the local
environment and this is the first time you are doing this on a given machine, beware that the stage two images will take some time to fetch. The terminal that you have run docker-compose-ui in will be visibly retrieving these images. There are some things that can go wrong:
error
message will be logged in the terminal you ran npm run dc-up
inMissing region in config
Error
logged. If you have configured the aws cli correctly then this is a red herringOn subsequent runs you should not see the above mentioned issues. If this concerns you, the timeout (found in the app-scanner) can be increased.
Depending on whether you are testing against a local copy of your containerised app or a copy hosted on the Internet will determine how you configure the cli (purpleteam). This example demonstrates the two options mentioned in step 3.
By specifying the local
environment, you are instructing the purpleteam CLI to use it’s config/config.local.json
, and communicate with the purpleteam back-end that you have already set-up and have running locally (step 4). If using the cloud
, the back-end is all taken care of for you. You will also need to specify the location of the SUT in the Build User config (Job) that you provide to the CLI. Examples of these can be found in the testResources/jobs
directory:
Locally cloned copy of NodeGoat:
The SUT details in your Job will be as follows:
"sutIp": "pt-sut-cont",
"sutPort": 4000,
"sutProtocol": "http",
NodeGoat running on the Internet via purpleteam-iac-sut:
The SUT details in your Job will be as follows, with <your-domain-name.com>
replaced with a domain you have set-up:
"sutIp": "nodegoat.sut.<your-domain-name.com>",
"sutPort": 443,
"sutProtocol": "https",
From the ~/Source/purpleteam/
directory, run the following command:
env NODE_ENV="local" npm start -- test
docker stats
:
docker container ls
If they haven’t been brought down:
purpleteam-s2-containers/app-emissary/
directory run the following command:
docker-compose down
purpleteam-s2-containers/selenium-standalone/
directory run the following command:
docker-compose down
If you want to keep the Stage Two containers running after a test run to inspect for any reason, simply change the app-scanner’s emissary.shutdownEmissariesAfterTest
config value to false
, rebuild the container and run.
If the Stage Two containers have not been brought down, you can also shut down the Stage Two containers with the docker-compose-ui UI. When it’s running, you can browse to http://localhost:5000, select the specific Stage Two container project and click the Down button, this is convenient as it brings all of the specific Stage Two containers down with one click.