Page 1 of 1

[1.1.21] Frozen on logon

Posted: Thu Feb 18, 2021 11:15 am
by distortions864
Headless server, Ubuntu 20

Someone logged in and the game stayed paused?

Code: Select all

18 hours, 49 minutes and 24 seconds
74813.065 ConnectionRequestReplyConfirm (type(ConnectionRequestReplyConfirm) ) from(IP ADDR:({109.145.138.195:61155}))
74813.065 Info ServerRouter.cpp:453: Replying to connectionRequest for address(IP ADDR:({109.145.138.195:61155})).
74813.065 Info ServerSynchronizer.cpp:600: nextHeartbeatSequenceNumber(2170154) adding peer(11)
74813.083 Info ServerMultiplayerManager.cpp:780: updateTick(4065884) changing state from(InGame) to(InGameSavingMap)
74819.450 Info RemoteCommandProcessor.cpp:242: New RCON connection from IP ADDR:({127.0.0.1:44132})
74841.327 Info ServerMultiplayerManager.cpp:1048: Disconnect notification for peer (11)
74841.327 Info ServerSynchronizer.cpp:619: nextHeartbeatSequenceNumber(2170998) removing peer(11).
74841.346 Info ServerMultiplayerManager.cpp:780: updateTick(4065884) changing state from(InGameSavingMap) to(InGame)
18 hours, 49 minutes and 24 seconds
18 hours, 49 minutes and 24 seconds
18 hours, 49 minutes and 24 seconds
18 hours, 49 minutes and 24 seconds
18 hours, 49 minutes and 24 seconds
Race condition?
the RCON command is me remotely running a commmand to see if anyone is online when user reported server frozen ( took a while to respond )
I tried to connect, and server did not respond. Other Factorio servers on same machine were fine.
It shouldn't be a file/io problem. there is 16GB of write buffer available, writing to a NVME SSD.
Also seems odd that it is peer 11, he was the only player.

Re: [1.1.21] Frozen on logon

Posted: Mon Mar 01, 2021 10:23 am
by Klonan
Can you provide a full server log/configuration?
I wonder... are you using the non-blocking save?

Re: [1.1.21] Frozen on logon

Posted: Fri Mar 05, 2021 4:08 am
by distortions864
Yes, non-blocking save is enabled.

Re: [1.1.21] Frozen on logon

Posted: Tue Mar 09, 2021 1:34 pm
by Klonan
Has this happened since? Do you have a way to reproduce it?

Re: [1.1.21] Frozen on logon

Posted: Sun Mar 14, 2021 1:04 am
by distortions864
For the moment, I disabled async save.
It happened very rarely, but it did happen a few times.

I will put it in my notes to see if there is anything specific correlated with it.

Re: [1.1.21] Frozen on logon

Posted: Mon Mar 15, 2021 1:06 pm
by distortions864
No idea what the cause is, but the freeze still happens (with async save off).
Only happens when a player is connecting.

log/saves/settings attached, Factorio 1.1.27.

Discord bot log:

Code: Select all

00-05-56-25 rbrunski: someone changed the belt chests back to passive providers?
[6:58 AM]
00-05-56-39 Schnei: not moi
[6:58 AM]
00-05-56-54 samarsen: sorry its me
[7:00 AM]
00-05-59-04 (SYSTEM) Welcome crazzyfoxx to the map!
00-05-59-04 crazzyfoxx joined (Member)
[7:03 AM]
00-06-00-04 FinnW disconnected. (Quit)
00-06-00-04 crazzyfoxx disconnected. (Quit)
[7:03 AM]
00-06-00-04 AskMcEscher: oof
00-06-00-04 AskMcEscher: yikes
[7:04 AM]
00-06-00-11 Distortions864 joined (Admin)
[7:04 AM]
00-06-00-22 Distortions864: weird
[7:04 AM]
00-06-00-24 AskMcEscher: that was quite a lag
[7:04 AM]
00-06-00-32 AskMcEscher: save bug i think
When I connected, I got stuck at 'saving map' for some time, the progress bar was at 100% though.
Totally possible I just have a bug/performance issue somewhere I guess?

Async save is off in this case, so not sure it is related.
The log from the async save ( first post ) does still look odd to me.
Its odd that in both cases, the issue is always when a player is connecting... never seems to be on autosave?

Scenario script:
https://github.com/Distortions81/M45-Factorio-Lua

Discord bot:
https://github.com/Distortions81/ChatWire

Re: [1.1.21] Frozen on logon

Posted: Mon Mar 15, 2021 4:57 pm
by kovarex
Can you reproduce it without custom scenario?

Re: [1.1.21] Frozen on logon

Posted: Tue Mar 16, 2021 11:10 am
by distortions864
I'll investigate as time permits.
I'll give the scenario a going-over, do some performance testing, and possibly set up one of our servers with no scenario.

Thanks for the responses!