The short version
What Railway has said so far.
Railway is a cloud where developers put their apps online, with more than 2 million users. On October 21, it said someone got into the systems that store its customers' environment variables and, for some of them, read the decrypted values.
Environment variables are where an app keeps its secrets, like API keys for OpenAI and Stripe, database passwords and logins for private code registries. Whoever reads them can act as your app.
Railway is asking every customer to replace every variable they store there and to check their logs. It says the impact may be limited to some users, and it will email the accounts it confirms were affected.
Railway paused every deploy for most of a day while it investigated. Builds came back for Enterprise and Pro workspaces overnight. Hobby, Free and Trial are still waiting, and new keys can't reach an app on those plans until they return.
Railway also cancelled every Railway API token, so anything that talks to the Railway API needs a new one.
- Treat every key in its variables as known to someone else until you replace it.
- Changing a value inside Railway doesn't stop the old key. Revoke it at the service that issued it.
- Check your AI and payment accounts for charges you don't recognise.
- Make a new Railway API token if a script or CI job uses one. The old ones stopped working at 12:39 AM ET on October 22.
How the day went
October 21 and 22, in Eastern time. Railway's status page uses UTC.
- Oct 213:24 AMRailway opens an incidentIts status page starts investigating an issue. Later updates say the dashboard, API and CLI are unavailable and new builds are paused in every region.
- 5:11 PMPostiz goes offlineNevo David, who runs the social media app Postiz, shuts it down for every customer after Anthropic and Stripe email him that his keys were compromised.
5:51 PMShow 5 more updatesHide 5 updates
- 5:51 PMRailway's warningRailway posts an urgent notice on X, linking its security disclosure.
- 6:10 PMBucket keys replacedRailway starts rotating every storage bucket credential. The new ones appear in the dashboard, but apps can't pick them up until deploys return. The resets show in audit logs under a Railway employee's name, which Railway says is expected.
- 8:22 PMDeploys still pausedRailway says it's still working to bring builds back and will publish guides for the next steps.
- 9:22 PMGuides publishedRailway posts step-by-step guides, like checking for unexpected OpenAI or Anthropic usage and rotating PostgreSQL credentials. Steps that need a redeploy still have to wait for builds.
- 11:32 PMEnterprise builds returnEnterprise workspaces can deploy again and apply the new keys they added in Railway.
- 5:51 PM
- Oct 2212:39 AMRailway API tokens cancelledRailway invalidates all of its own API tokens. Anything using the Railway API stops working until it gets a new token.
- 3:11 AMPro builds returnPro workspaces can now finish the guide steps that need a redeploy. Hobby, then Free and Trial come next.
- 6:15 AMDeploys stuck in a queueMany deploys wait longer than usual and some builds fail more often. Railway asks people not to start the same deploy twice. Hobby, Free and Trial still can't build.
What to do if your app runs on Railway
In this order. Each step links to the official docs.
- 1
List every secret your app uses
Open the Variables tab of each service, or your own .env file, and write down every key, token and password. Include database URLs and private registry logins.
- 2
Revoke each key where it was made
Make a new key in the service that issued it (OpenAI, Anthropic, Stripe, GitHub, your database) and delete or expire the old one there. A new value in Railway alone leaves the old key working.
- 3
On Stripe, expire the old key now
When you rotate a Stripe key, the old one keeps working for up to 7 days unless you pick Now as its expiration. For a leaked key, pick Now.
- 4
Decide what can go down
On a plan that can't build yet, a revoked key breaks a live app that still uses it until you can redeploy. I'd still revoke AI and payment keys right away. A few hours offline costs less than a stranger spending on your account.
- 5
Check your bills and logs
Look at usage on every AI account from October 21, and from October 16 if you saved keys in OpenCode Console. Read your service logs for anything you don't recognise.
- 6
Set a hard spending limit on AI accounts
In the Claude Console, set a spend limit on the Billing page and requests stop when you reach it. In OpenAI, a project's monthly spend limit only sends alerts unless you enforce it as a hard limit.
- 7
Make a new Railway API token
Railway cancelled every Railway API token. If a script, CI job or tool talks to the Railway API, make a new token in your account settings and swap it in.
- 8
Redeploy with the new keys
Enterprise and Pro can redeploy now. Hobby comes next, then Free and Trial. Start each deploy once and let it wait in the queue, since Railway says repeats make the wait longer.
Railway's own guides
Railway published these at 9:22 PM ET on October 21. Steps that need a redeploy work now on Enterprise and Pro, and wait for builds on Hobby, Free and Trial.
- 1
Check your AI accounts for usage you didn't make
Railway wrote one each for OpenAI, Anthropic, OpenRouter and Grok.
- 2
- 3
Rotate your database credentials
Railway has a guide for each database it hosts.
- 4
Redeploy buckets and reconnect GitHub
Railway reset storage bucket credentials, so buckets need a redeploy and GitHub needs reconnecting.
Why someone wants your AI keys
Railway says it isn't the only target.
Railway believes this is part of a series of attacks on platform companies, aimed at getting into AI providers. It has told those providers to revoke keys.
On October 6, Fireworks AI, which runs AI models for developers, found that someone had used internal Fireworks credentials to read environment variables in some customers' evaluation jobs, including API keys and tokens.
On October 16, OpenCode Console, where people could save their AI provider keys, leaked keys for 1.4% of its users. People reported charges they didn't make on OpenAI, Anthropic and Google, one totalling $1,700. Keys kept in the OpenCode app on people's own computers weren't affected.
A working AI key is spending money. Whoever holds it runs models on your bill, so any service that stores a lot of them is worth breaking into.
Posts and coverage
What we read for this page, in the order it happened.
Fireworks AI's bulletin
Fireworks' own write-up of the October 6 incident and what customers should rotate.
fireworks.aiTechCrunch on OpenCode
Published three days after the October 16 breach, on how OpenCode's Console leak ran up charges on users' AI accounts.
techcrunch.comThe Hacker News on OpenCode
Which keys leaked, the charges users reported and why local keys were safe.
thehackernews.comPostiz founder on X
Nevo David shuts Postiz down after Anthropic and Stripe email him about his keys.
x.com/wickedguroThe Hacker News thread
Developers comparing notes, including keys only stored in Railway being used.
news.ycombinator.comKeeping keys safer from here
Small habits that shrink the next leak.
Give each app its own key, so one leak means replacing one key. Stripe recommends restricted keys, which carry only the permissions an app needs, over full secret keys.
Stripe can also lock a key to your own servers with an access policy, and it notifies you when it blocks a request from somewhere else.
Railway has sealed variables, whose values reach your app but never show in the dashboard or the API. Railway hasn't said whether sealed variables were safe in this breach, so replace those too.
Set spending limits on AI accounts before you need them.
Read Railway's disclosure- How many customers were affected. Railway says the impact may be limited to some users.
- How the attacker got in.
- Whether sealed variables were read.
- When Hobby, Free and Trial builds come back.
