yet another smart train System & Factory (Img heavy)

[cid0rz]

hello fellow factorians. I've been very busy in factorio and outside recently which prevented me form posting this before. I had also a nice migration to 0.15 so congratulations to the devs and the modders for making us so happy^^. In this case the mod I use is of course Choumiko's smarter trains and as a reference when starting I read siggboy's build and even get the blueprints and play a bit with them but I'm a bit slow so I prefer to do things on myself from the beginning in order to learn. Basically I like mining and building millions of iron plates but the satisfaction of developing a system like this is one step further so I encourage you to try. My following discussion will be descriptive without going into much details. I'll provide the blueprints so you can investigate and try around but the configuration if you intend to use it for production can be tricky sometimes. Anything you need or ask I'll try to answer.

So I wanted to start a new factory, really nice one, to have high availability of finished products. Then I thought about doing it train based with a delivery system and there it all gone crazy. I spent like 3 days cabling uncabling testing.. I had done some stuff with combinators but this was not easy.

The Smarter trains mod has to be installed and all the trains that we intend to operate in the same circuit must belong to the same line. On this line all the stations must be listed, my recommendation to speed up is to put all the providers in the beginning of the line and all the requesters at the end. If you know how many stations youll have you can create "dummy" stations for the future provider's but that is too much detail.

Let's see the Depot:

depot

depot1.jpg (362.1 KiB) Viewed 7949 times

basically it consist on a 1st sage polling area, where if something is demanded (negative numbers) the memory cell will register the order and activate stage2. On stage 2 it will poll all the stations to see if someone can provide what it is required, if there is, it will be registered on the second memory cell. Once these 2 cells are booked, the stops are booked. Beware! if you don't send the trains to those stations, you'll have problems. Finally it will calculate internally how many trains does it need to deliver the goods depending on their stack sizes. And the third memory cell will deliver actually the trains. When it is emptied, the polling system starts again. The Yellow aprt is the 1st in polling area, the memory cells are numbered 123, the calculation of number of trains is done in the green part, the blue constant combinators hold the stack sizes.

The signals that are used for communication between stations are unusable for transporting items, this means my system cannot transport diesel locomotives, train stations, wooden chests or cargo wagons. this could be changed masquerading those signals i did it in the depot in fact since the last memory cell is a bit problematic. But for now these i thought wouldn't be very useful to transport by rail ^^


let's make an example and talk about it so I went to my factory and get some plate running. The depot would start polling with the train station signal, it will go increasing the station signal till a certain value (for me now is below 50). When it get's some request it will star searching for a provider that has this quantity available. The poll will be in this second round with wooden chests. Then the internal calculation and the final order. For our example I removed all the trains from the Depot so the polling will stop (why to poll if you don't have trains to deliver right?) and we can see the booked order and understand it:

depotloaded

depot2.jpg (738.41 KiB) Viewed 7949 times

We can extract valuable information here, The requester R20 Asked for 7k Iron plate, it needs a 4 wagon train. The provider P15 can deliver up to 69k iron plate atm so it will deliver it (number of wagons is checked in the provider when it is asked) and one single train can transport it with 8.9k remaining free capacity. In case multiple trains are needed the total amount requested will be divided by the number of trains and the requester and provider will know and will stay booked for the whole order.

A couple of sidenotes, in theory you can have different trains with different number of wagons, this is not fully implemented now and I don't see the real need but i think with a bit of tweaking it cound be properly managed. i made all the stations so they can book multiple orders but when cabling them for normal use i modified them so they can only accept one and wait for the deliver to be finished to order another. this makes sense since we can deliver multiple trains and otherwise even if what it is promised is taken into account at the outpost, could lead to more requests tan desired. There are many details and it is hard to go through them, I realize while I write that many things are left to be described but as I said it is not a full detailed guide.

So let's ride to our provider, the depot will pass the P15 to the train and we¡ll be on our way. before arrriving we'll see the Provider ready to serve:

Provider

provider.jpg (711.67 KiB) Viewed 7949 times

In the provider we can see 1 train coming, to take 7k ironplate to go to R20. The signals W15 and Y20 are just booking signals for the corresponding requester and provider. As they are sent together the'll be on both stations. I have a loading circuit and unloading circuits on my stops, you can see them on the blueprints and I''ll not cover them here. So once loaded R20 is passed to the smarttrainstation and let's go to the requester:

Requester

requester.jpg (769.57 KiB) Viewed 7949 times

There we can see..... WAIT! xDD yees I knooow, So we can see our order, plus some other order. What happened is i went to my dreamed factory, that has a special requester. This special requester will handle requests of 8 materials needed in the factory. Anyway the improtan numbers ie. the quatities are ok.

let's talk a bit about the config of the trains:
On the depot they wait for signal P
on any provider they wait for signal R
on any requester they wait for signal D (alwais = 1 a.k.a. the Depot) AND empty cargo inventory

That's why you can see the lamp on in the requesters when they are booked but not on the providers since they have to load first and sometimes you don't want them full. Always some excess of material will be transported, that's why I usually make the standard requests smaller than one train.

Regarding connecting the requesters and providers, please add them to the line in smart trains before connecting anything, even electricity for the requester will lead to malfunction. Once they have their smarstation number asigned then you can connect them. Also remember each time you add a station you have to configure the line so the train will be directed to Signal#

Before leaving let me show you the multirequester station (in reality is a small polling system similar to the one used in the train system but just for the factory), it can book a configurable amount of trains (for me if 3 trains are on their way it waits) then a big filtering belt to feed the buffer that will feed the central bus. There is a pic of our order booked on the factory buffer. Then i have a system connecting all the factory outputs and a smart loading station to make trains with the quantities you configure, this if is interesting for somebody I can share. I hope you enjoyed and please let me know any idea to improve it or to work on.

order booked in buffer

Our example order booked inside the factory

orderbooked.jpg (743.12 KiB) Viewed 7949 times

main filter form the air

mainfilter.jpg (974.88 KiB) Viewed 7949 times

main bus

mainbus.jpg (890.29 KiB) Viewed 7949 times

The factory output for configurable train loads

factoryoutput.jpg (859.33 KiB) Viewed 7949 times

The factory form the air

factory.jpg (962.67 KiB) Viewed 7949 times

[cid0rz]

Here the blueprints since too many attachements.

Edit: for the people downloading the Blueprints, on the yellow part of the Depot, leftmost there is a constant combinator with white20, if you turn this on the polling will stop and you can solve any mess happened with a lot of patience

[doxsroxs]

Hi again, what I meant in the Smart trains thread is that any system that loops through a station list etc. will have a very limited maximum throughput since you can only loop through so many stations per second.
A very large system where you have hundres of stations might have to match and dispatch many trains per second.

My thoughts for a peer to peer system is to try and have the requester stations contain the logic, by spreading the computational load to each station the maximum throughput of the system will also be increased.
Im still experimenting, but if possible I want to use only the material signal for matching stations. (possibly combined with a central clock)

For example, provider stations signal when they have a train loaded and ready with iron.
When a station needs iron it will look for that signal on the wire.
The requester stations sends a large negative iron signal, this initiates matching mode for iron in the iron providers. (should be time limited to x number of ticks)

This is where I am not quite sure how to do this, if there is only one active iron provider it is easy, however, if there are 5 one will need to be selected for delivery.
Either Ill have to use a random number to add delays before each station replies and then picking the first station that responds (assuming no collisions). (possibly to slow solution)
Or Ill need some way to mathematically deduce which station is to respond first with its station ID.
Once that is known the rest should be no problem. Just send a confirmation and store that you have one trainload incoming so you dont request more unless you can fit more.

My main goal with the build is to try and use the material signal itself for matching to enable the system to match all materials at once.
My secondary goal is to keep the matching time as low as possible in ticks since this sets the maximum throughput in the system for each material.

[doxsroxs]

Dont forget to pester the devs about the ability to send trains using combinators! :p
viewtopic.php?f=38&t=46557&p=270556#p270556

[cid0rz]

Hi again, what I meant in the Smart trains thread is that any system that loops through a station list etc. will have a very limited maximum throughput since you can only loop through so many stations per second.
A very large system where you have hundres of stations might have to match and dispatch many trains per second.

My thoughts for a peer to peer system is to try and have the requester stations contain the logic, by spreading the computational load to each station the maximum throughput of the system will also be increased.
Im still experimenting, but if possible I want to use only the material signal for matching stations. (possibly combined with a central clock)

For example, provider stations signal when they have a train loaded and ready with iron.
When a station needs iron it will look for that signal on the wire.
The requester stations sends a large negative iron signal, this initiates matching mode for iron in the iron providers. (should be time limited to x number of ticks)

This is where I am not quite sure how to do this, if there is only one active iron provider it is easy, however, if there are 5 one will need to be selected for delivery.
Either Ill have to use a random number to add delays before each station replies and then picking the first station that responds (assuming no collisions). (possibly to slow solution)
Or Ill need some way to mathematically deduce which station is to respond first with its station ID.
Once that is known the rest should be no problem. Just send a confirmation and store that you have one trainload incoming so you dont request more unless you can fit more.

My main goal with the build is to try and use the material signal itself for matching to enable the system to match all materials at once.
My secondary goal is to keep the matching time as low as possible in ticks since this sets the maximum throughput in the system for each material.

Welcome my friend!

I was writing crap for 1 hour you can find in the end. Then I realized... If you want to have 1 train for every couple of stations (this has a very high throughput) go for the traditional solution, don't make it smart system. Having a random number of providers and requesters and trains it is a very complex and probably quite specific build, in the sense probably you'll have to make the turns for your different resources. Then transmitting multiple signals of multiple resources at the same time is painful. You can make then some kind of numeration of the stations form the same resource so to choose the closest to your destination, you can make relative distance calculations... many things, all of them really overcomplicated.How many stations need something usually?

I will make some trials on the distributed system but all the ideas when thinking about it become complex very fast. I'll see if trying some inspiration come

crap

So in your specification for the system you will have exactly 1 train per provider that will be busy delivering something or at the station, loading or ready to go, right? Then the problem of managin the provider, get's easyer, it will simply get ready to accept orders when the train is loaded so the provider would be set up.

Then we would need a central clock, when clock=10 for example it will ask for requesters that need something. Here comes the many problems:
If more than one requester want the same resource... will you synchronize them with the central clock so they have different turns? This will make you loose all the advantage of a distributed system.
Let's try to simplify so, the amounts transported will always be one train, so the requester just needs to say hey I need something and something will be always one and only one train. Even then we'd need to know what requesters need something in order to dispatch the train

if you have multiple providers with the same resource, you'll need more than one signal and the resource signal will get added with all the ready providers. This as you say can be solved delaying the answers and creating turns with the central clock, For me this solution is not very good. I can think in masquerading all the signals for the same resource and making turns for the resource, with a base of 200 stations it should be better since there are not 200 types of goods to transport so you could do for the resource signal let's say iron, convert it to consecutive signal numbers depending on the distance. Another limit for this build i'm realizing now is you'll use 1 train for provider well you can have a depot of trains in the provider, so ok xD. Let's say this "can be solved".

[cid0rz]

I was thinking again, If you don't determine what your trains do, so you send a train from a provider to a requester, and then? it will go back? if you don't have 1 train for each provider this does not make sense but neither does that the train waits in the requester forgetting refueling and that then requester should be able to dispatch. How is exactly the way you want your system to work, this is the most important, if it is not well specified is very difficult to find solutions

[doxsroxs]

Building a normal non smart train network is no fun, so no, I wont do that. Also once fleshed out this is way more time efficient. Even with the old Siggboy setup I could plop down a mining setup or a new requester station very fast.
This is needed when you grow big to ensure you dont spend 100% of your time configuring trainlines or messing with configurations when adding stations.
It also has to handle all resource types and be flexible enough to handle mods where resources types are added.

The simple solution for empty trains would of course be to send unused trains to a central depot and dispatch them from there to requesters that need them.
The problem with that is that it does not scale with size and even a large depot can become a bottleneck.
Ill probably setup the system like that to begin with, but I also envision a smarter system where the requester station will send the train to any provider station that needs an empty train once it is unloaded and only send it to the depot if no trains are needed.

Fuel has not been an issue in the past and I dont really think it will be this time either.
Refueling them at major stations and at the depot has been enough. If possible Ill add a check for that as well.

When it comes to the matching system my current idea is to use some sort of binary adress table to be able to use single bits as station adresses, then convert the bit to a proper adress number.
Ill see if I can get my thoughts together on this in a bit more ordered fashion and get back with a better explanation.
Assuming I can get it to work outside my head that is... :p
This might of course add other limitations on the system, so its all a trade.

My main priority is on throughput, this has to scale to huge levels, and to achieve that matching times must be as low as possible. A few ticks would be awesome.
In an absolute emergency I might build the system to require both the green and red wire, but I hope to achieve this using only one signal wire.

[cid0rz]

In a project you have requirements, in your case is smart system and high throughput. In factorio regarding programming anything is possible but usually you'll have to get to a compromise where the system complexity is reasonable for the purpose. I think the main problem of the distributed system is the synchronization and having to process many signals that can overlap simultaneously. I'll keep thinking about it but I don't see the solution to this problem. If you don't mind please share with us your progress ^^

[doxsroxs]

In a project you have requirements, in your case is smart system and high throughput. In factorio regarding programming anything is possible but usually you'll have to get to a compromise where the system complexity is reasonable for the purpose. I think the main problem of the distributed system is the synchronization and having to process many signals that can overlap simultaneously. I'll keep thinking about it but I don't see the solution to this problem. If you don't mind please share with us your progress ^^

Ill post my own thread once I have sketched things out a bit more, I will probably need help with some parts to make this work :p

[cid0rz]

Hello everybody again. I've recently discovered fCPU mod for Factorio and I couldn't let go the opportunity of revisiting the train system and porting it to this new technology . Now the circuits are much lighter and less obscure even if still a bit complicated. I think with fCPU is easy to do very neat control schemes. Smarter trains is also needed to be able to dynamically change the train schedule of the train.

I have prepared a mock-up circuit that uses wood to test the system.

mockup areas

mockup_areas.jpg (202.18 KiB) Viewed 4941 times

In the picture you can see in red the main server/brain of the system, it will poll the stations and organize the trains. Then in green you can see 2 constant combinators to select which station will be active. There are 2 providers and 2 requesters on the mock-up, with those constant combinators you can switch which station works and is what i used to test the system. Finally in blue there is a constant combinator that allows for an easy way of adding more wood to the providers when desired. It is a mockup to play with, so without feeding wood to the providers the trains will stay on Depot since the request cannot be fulfilled.

VERY IMPORTANT WARNING: This is a toy circuit and the system has not been tested in depth, so you probably will encounter errors and problems. If you don't like that please wait for a future version or beware. I have configured the system for 100 stations total and if you plan to use it in your factory is probably a good idea to talk to me. There are some settings and things to be tweaked if you are going to use it for a "real world" application . Id gladly come and help configure. If many people are interested I will consider making a short guide.
It is also very important if you are going to use the mock-up YOU NEED TO CONFIGURE it. You need to create a line with all the stations in it and the depot must be the first station of the list. Then in the line settings you need to check the tick for the trains to go to signal# as shown in the following picture. With this should be sufficient to test it.

smart trains line config

stconfig.jpg (506.31 KiB) Viewed 4941 times

And here some details on the components:

station

station.jpg (876.25 KiB) Viewed 4941 times

The stations have 2 fCPU's, the main one that is marked on the picture and the secondary one that acts when station is waiting for a train.

feeder

feeder_detail.jpg (846.35 KiB) Viewed 4941 times

You just need to set a negative quantity on the constant combinator and turn it on for a short time. This will release that quantity of wood from the feeder to the provider that is currently active.

main server

server.jpg (786.99 KiB) Viewed 4941 times

You just need to hit play on the main fCPU of the server to get it going.

Here is the book (OUTDATED, see below) with all the BP's. Let me know if you come up with something similar/improvements/etc .

Cheers!

PD: There are new features planned on fCPU that will simplify a lot the system and maybe make some new features easier to implement so I plan on working on that soon.

[cid0rz]

Hi everyone,

let me present v2.0 of the smart booking system with fCPU and Smarter trains mods. It is just a small set of improvements over version 1 that makes much faster and cleaner and also easy to use and understand. I have also written a lot of comments on my fCPU programs so they are easy to follow/hack. Id be very happy to see any application or development . I've created blueprints with two mock-ups, the bigger one could serve as a stub for blueprints for a real production system. I've also created a git repo with the commented programs and the blueprints.

Mockup operation and Smarter trains configuration required remain the same. If you have problems please contact me.

Happy wiring!