Deploy

Troubleshooting Workspace Deployment

Updated 26 Oct 202612 min read

This guide helps you find out why a Workspace failed to deploy, update, or delete, and how to fix it. Most failures are caused by something in the AWS account the Workspace is deployed into (permissions, quotas, or leftovers from an earlier Workspace), and you can fix them yourself. Where only autobotAI can fix the problem, the guide says so.

Prefer to troubleshoot with an AI assistant? Download the Workspace troubleshooting file for Amazon Q and load it into Amazon Q Developer. See Troubleshoot with Amazon Q.

How a Workspace Deployment Works

A Workspace deployment runs in four phases. Knowing which phase failed tells you where to look.

PhaseWhere it runsWhat happens
1. Region preparationautobotAIAfter you click Create, autobotAI copies the Workspace templates and container images to your chosen AWS region. You see Preparing Workspace Region...
2. Stack creationYour AWS accountYou click Launch Stack and AWS CloudFormation creates the Workspace stack (AutobotAI-Workspace) and its nested stacks. This takes 30–45 minutes.
3. Registration and frontendYour account, then autobotAIWhen the stack finishes, it registers itself with autobotAI. autobotAI then deploys the Workspace web app at https://<workspace-name>.autobot.live.
4. Updates and deletionYour AWS accountUpdate Version gives you an AWS CLI command that updates the stack. Delete removes the Workspace.

What the Workspace status means

The Status column on the Workspaces page shows where the Workspace is:

StatusMeaningIf it stays like this
stack template generatedThe template is ready. Waiting for you to launch the stack.The stack was never launched, is still running, or failed. See Find the failure in CloudFormation. If the stack finished, see Stack finished but the Workspace is stuck.
stack creation complete / frontend deployment in progressThe stack finished and registered. autobotAI is deploying the web app.Wait up to 15 minutes, then contact autobotAI support.
create complete / update completeThe Workspace is running.None
create failedThe stack was deployed in the wrong AWS account, or the web app deployment failed.See Stack finished but the Workspace is stuck.
workspace update requestedAn update command was generated and is waiting to be run.Run the command, or use Cancel Version Update from the ⋯ menu.
update failedThe update of the web app failed.Check the stack for an update failure, then use Update Version again.

Find the Failure in CloudFormation

When a stack fails, CloudFormation records the exact reason. Start here for any failure in phase 2 or 4.

  1. Sign in to the AWS CloudFormation console in the same account and region as your Workspace.
  2. Open the AutobotAI-Workspace stack and go to the Events tab.
  3. Look for the first event (the oldest one) with a status ending in _FAILED. Later failures are usually just CloudFormation cancelling other resources after the first one failed.
  4. If the failed resource is a nested stack (PersistStack, NewDisposableStack, or DefaultIntegrationStack), the real reason is inside it. Click the nested stack's Physical ID link and repeat step 3 in its Events tab.
  5. Read the Status reason of that event and find it in Common Failures and Fixes.

You can also list the failures from AWS CloudShell (in the Workspace region):

bash
aws cloudformation describe-stack-events --stack-name AutobotAI-Workspace \ --query "StackEvents[?contains(ResourceStatus, 'FAILED')].[Timestamp,LogicalResourceId,ResourceStatusReason]" \ --output table

Note: CloudFormation deletes failed nested stacks during a rollback. To still see their events, open the Deleted filter on the CloudFormation Stacks page.

Read the Logs

Workspace setup steps run as AWS Lambda functions. When one fails, its status reason says which step failed and names the CloudWatch log stream with the full error, for example:

text
Connecting to DocumentDB failed: ServerSelectionTimeoutError: ... (full traceback: CloudWatch log stream 2026/10/13/[$LATEST]abc123)

Open CloudWatch Logs in the Workspace region and find the log group. Every Workspace log group is named /aws/lambda/abAI-<function>-live-<workspace-name>. The ones that matter during deployment are:

Log group (replace <ws> with your Workspace name)What it does
/aws/lambda/abAI-PostWSLambda-live-<ws>Creates the administrator user, subscription, and default integrations
/aws/lambda/abAI-BotMigrationV2-live-<ws>-v2Runs database migrations and syncs the integration catalog
/aws/lambda/abAI-PostMigrationLambda-live-<ws>Applies DocumentDB settings after the cluster is created
/aws/lambda/abAI-abAISecretManagerV4-live-<ws>Creates the Workspace configuration secret
/aws/lambda/abAI-create-ecs-role-live-<ws>Creates the ECS service-linked role
/aws/lambda/abAI-copy-file-live-<ws>Copies integration templates and empties buckets on delete
/aws/lambda/abAI-RegisterWorkspace-live-<ws>Registers the finished Workspace with autobotAI
/aws/lambda/abAI-ENICleanup-live-<ws>Removes Lambda network interfaces on delete

Common Failures and Fixes

Region preparation failed

These failures happen in autobotAI before you launch the stack, so you can't fix them in your AWS account.

What you seeCauseFix
Region Preparation Failed with ECR bootstrap failed for ...Copying the Workspace container images to your region failed.Contact autobotAI support.
Region Preparation Failed with S3 file copy failed: No files found ...The selected Workspace version isn't available.Choose another version, or contact autobotAI support.
Region Preparation Failed with Failed to grant ECR accessautobotAI couldn't give your account access to the container images.Contact autobotAI support.
Preparing Workspace Region... for more than 20 minutesRegion preparation timed out.Contact autobotAI support.
Bootstrap failed previously.Region preparation failed earlier. The original error is no longer shown.Contact autobotAI support.

Tip: Workspaces in US East (N. Virginia) don't need region preparation, because the templates and images already live there.

Stack failed: account and permissions

Status reason containsCauseFix
Source image ... does not exist or Lambda does not have permission to access the ECR image for an image in 023425967463.dkr.ecr...The stack was launched in a different AWS account or region than the one you entered when creating the Workspace. autobotAI only gives that one account access to the images, in that one region.Delete the stack. Launch it again in the account and region shown on the Workspaces page.
Requires capabilities : [CAPABILITY_IAM] or [CAPABILITY_NAMED_IAM] or [CAPABILITY_AUTO_EXPAND]The acknowledgement boxes at the bottom of the Create stack page weren't ticked.Delete the failed stack and launch it again, ticking all the boxes.
is not authorized to perform or AccessDeniedThe IAM user or role launching the stack doesn't have enough permissions.Launch the stack with an IAM principal that has AdministratorAccess.
with an explicit deny in a service control policyAn AWS Organizations service control policy (SCP) blocks the region or a service the Workspace needs.Ask your AWS administrator to allow the region and services (Lambda, ECS, DocumentDB, VPC, IAM, Cognito, API Gateway, SES) for this account.
Creating the ECS service-linked role (AWSServiceRoleForECS) failedThe account can't create the ECS service-linked role, usually because of an SCP.Create the role once by hand. See Check ECS Service-Linked Role.

Stack failed: AWS quotas and region support

Status reason containsCauseFix
The maximum number of VPCs has been reachedThe region already has 5 VPCs (the default limit). The Workspace creates one.Delete an unused VPC, or request a quota increase.
The maximum number of addresses has been reachedThe region already has 5 Elastic IP addresses (the default limit). The Workspace creates one for its NAT gateway.Release an unused Elastic IP, or request a quota increase.
NatGatewayLimitExceededThe Availability Zone already has the maximum number of NAT gateways.Delete an unused NAT gateway, or request a quota increase.
DocumentDB resource fails with Cannot find version or instance class ... not supportedThe DocumentDB version or instance size isn't offered in that region.Deploy in another region, or contact autobotAI support.

Stack failed: a resource already exists

You can run one Workspace per AWS account and region. Deploying a second one into the same account and region, or deploying again before an old Workspace is fully deleted, fails because names are already taken.

Status reason containsCauseFix
Stack [AutobotAI-Workspace] already existsA stack with that name is still in the region, often an earlier failed one in ROLLBACK_COMPLETE.Delete the old stack, then launch again.
Domain already exists or Domain already associated with another user poolAnother Workspace's Cognito domain exists in this account and region.Delete the other Workspace first, or deploy into another account or region.
already exists for a log group (/aws/lambda/abAI-...) or an IAM role (abAI-...)Leftovers from an earlier Workspace with the same name.Delete the leftover log groups or roles that end in -live-<workspace-name>, then launch again.
Secret live/autobotAI/config is scheduled for deletionThe configuration secret of a deleted Workspace is still in its recovery period. The Workspace normally reclaims it by itself. When it can't, the message tells you what to run.Run the command from the message. To start fresh: aws secretsmanager delete-secret --secret-id live/autobotAI/config --force-delete-without-recovery

Stack failed: a Workspace setup step

These reasons have the form <step> failed: <error> (full traceback: CloudWatch log stream ...). Open the log stream named in the message (see Read the Logs) for the full error.

Step in the status reasonCommon causeFix
Loading the workspace config secret / Connecting to DocumentDBThe setup function can't reach DocumentDB, usually because network routing in the Workspace VPC was changed or blocked.Make sure no SCP or network policy changes the Workspace VPC, then delete and launch again.
Creating the super admin Cognito userThe Administrator Email is invalid.Create the Workspace again with a valid email.
Saving the super admin in DocumentDBThe user setup function returned an error.Check /aws/lambda/abAI-autobotCognitoPost-live-<ws>, then contact autobotAI support.
Creating the workspace subscription / Creating the default integrations / Creating the default AWS integrationWriting initial data failed.Contact autobotAI support with the error.
Running database migrations / Syncing the integration catalogA database migration failed or timed out.Contact autobotAI support with the error.
Applying DocumentDB post-migrationDocumentDB settings couldn't be applied. This is often outbound internet access through the NAT gateway being blocked.Make sure the NAT gateway and its routes are not blocked, then contact autobotAI support.
Copy failures from the copy-file functionThe account can't read the public autobot-ai S3 bucket, usually because of an SCP or S3 policy.Allow s3:GetObject on arn:aws:s3:::autobot-ai/*.

Note: Verifying the Administrator Email in Amazon SES never fails the stack. If the verification email doesn't arrive, see Verification link expired?.

Stack finished but the Workspace is stuck

What you seeCauseFix
The stack is CREATE_COMPLETE, but its Outputs tab shows WorkspaceRegistration starting with NOT REGISTEREDThe stack couldn't register with autobotAI. Usually the one-time deployment key has expired: it's valid for 3 hours after you click Create. Outbound internet access may also be blocked.Delete the stack and the Workspace, create the Workspace again, and launch the stack within 3 hours. If outbound access is blocked, allow HTTPS to api.autobot.live.
The Workspaces page shows an event Stack Deployed in Incorrect AccountThe stack was launched in a different account than the one you entered.Delete the stack and launch it in the account shown on the Workspaces page.
create failed or update failed a few minutes after the stack finishedautobotAI couldn't deploy the Workspace web app.Contact autobotAI support.

Update failed

What you seeCauseFix
The stack update finishes, but the status stays workspace update requested and the stack's WorkspaceRegistration output starts with NOT REGISTEREDThe update command contains a one-time key that's valid for 3 hours, and it expired before the command was run.Use Cancel Version Update, then Update Version again to get a new command, and run it straight away.
Stack status UPDATE_ROLLBACK_COMPLETEThe update failed and CloudFormation rolled back. The Workspace still runs the previous version.Find the first failure (see Find the Failure in CloudFormation), fix it, then use Cancel Version Update and update again.
Stack status UPDATE_ROLLBACK_FAILEDCloudFormation couldn't roll back a resource.In the CloudFormation console choose Stack actions → Continue update rollback. If it fails again, skip the resource that failed (--resources-to-skip) and contact autobotAI support.

Delete failed

What you seeCauseFix
DELETE_FAILED on a subnet, security group, or the VPC, mentioning has dependencies or a network interfaceLambda network interfaces (ENIs) were still attached when the stack tried to delete the network. The Workspace removes them automatically, but AWS can take longer than expected to release them.Wait 15–30 minutes and delete the stack again. If it still fails, delete the available network interfaces in that VPC from the EC2 console and retry.
DELETE_FAILED on an S3 bucket (The bucket you tried to delete is not empty)A Workspace bucket still had objects.Empty the bucket in the S3 console and delete the stack again.
The next deployment says a log group already existsLog groups were left behind after the delete.Delete log groups that end in -live-<workspace-name> and deploy again.

Retry a Failed Deployment

  1. In the CloudFormation console, delete the failed AutobotAI-Workspace stack and wait for DELETE_COMPLETE.
  2. Fix the cause you found above.
  3. On the autobotAI Workspaces page, delete the failed Workspace and create it again. This gives you a fresh deployment key, which is valid for 3 hours.
  4. Click Launch Stack and create the stack in the same account and region you entered.

Troubleshoot with Amazon Q

autobotAI provides a troubleshooting file written for AI assistants. It describes how a Workspace is built, where to look, and the known failures. Amazon Q Developer can use it to investigate the Workspace in your account and explain what went wrong.

Download the Workspace troubleshooting file

Load it into Amazon Q Developer in one of these ways:

  • In your IDE (VS Code or JetBrains): save the file as .amazonq/rules/autobotai-workspace-troubleshooting.md in any project folder. Amazon Q uses files in that folder as context for every chat in the project. You can also add it to a single chat by typing @ and choosing the file.
  • In the Amazon Q Developer CLI: start q chat, then run /context add <path-to-the-file>.
  • In the AWS Console chat: paste the parts of the file that match your problem into the chat.

Then ask, for example: "My autobotAI Workspace stack named AutobotAI-Workspace failed in us-east-1. Find out why."

Safety: The file tells Amazon Q to run read-only commands only, and to ask you before it changes or deletes anything. Review every command before you allow it.

Contact Support

If you still can't find the cause, contact autobotAI support with:

  • The Workspace name, AWS account ID, region, and version
  • The Workspace status and its events from the Workspaces page
  • The first failed CloudFormation event, with its full status reason
  • The CloudWatch log stream named in that reason, if there is one
Need a hand with this?Our engineers can walk through it with you on your own stack.