Random Ramblings #11

[kovarex]

The main reason to write random ramblings today is, that I have free time to do it, you will understand why.

OCZ sucks
So here we are, few days until the release. I should work hard to get everything done, and instead I'm watching Factorio streams(http://www.twitch.tv/myrathi) and relaxing. Why? Because my OCZ Vertex 3 drive started to freeze whenever I try to compile or do something performance heavy. The last time this started to happen, I could just update the firmware (2.25) and it was solved. But this time there is nothing I can do. I read through the forums (both official and other) for few hours and I found out, that the company burnt lot of cash in the past few months, is near the bankruptcy (I'm not surprised), and lot of people are having the similar problems as I do. On the top of that, the company tries to hide some of the problems, they delete answers of customers, release firmware updates often, or even revert existing updates when they find out they have even more bugs. Even the firmware updating software has bugs (You have to plug off all other hard drives, and when it doesn't work because of that, without any error message, they blame YOU for doing it wrong etc. etc.).
So I need to go and buy brand new ssd tomorrow (Any ideas what should I buy?) and spend the whole day by installing and configuring my computer, yay.

Things are getting tight
Tomas is working on the roboports to work properly, and it still needs few days to be finished, we will be really happy if we start bugfixing 2 days later than we planned to. Albert is working on the roboport, and he hopes to find some free time to finish some smaller things for the 0.8. So the release will be deadly as usual

Questions
I created ask.fm account at http://ask.fm/FactorioGame, I never used this platform personally, but I have seen some of the answers of my friends interesting.

[Gammro]

Too bad about the SSD, and as for a new one: I heard some really good things about the Samsung 840 EVO series. As usual with SSD's, the bigger the drive, the faster it is.

[Dysoch]

i got the samsung 830 serie ata ssd drive. it works like a charm (as does nearly everything i have from samsung) They deliver great products, which are durable. So i suggest getting an Samsung

cant wait for 0.8. hope to see it as soon as possible

[FrozenOne]

Kovarex,
I was talking with devs of indie FarmFortress, and as they are planning to release an update soon too and i like both games, they would like to exchange some advertisements, perhaps just notice on forums, maybe some mention on website or ingame, i think both games have similar fanbase and both would profit from knowing about each other. Are you for it? I'd give them your mail.

[Balinor]

Samsung and Sandisk seem to be getting the best reviews currently for SSD's. I'd be happy with anything from either company currently.

[Shirow]

I'll second the Samsung 840 EVO Recommendation.

Samsung since they entered the market have been generation after generation becoming one of the more dominant forces in the SSD industry (primarily due to them manufacturing both their own Flash memory, and Controller used), and have gained a lot of market share (consumer and enterprise) for good reason.

I'd highly recommend the EVO for two reasons 1. It has the best Price/GB ratio of any drive right now except possible the Crucial M500's (Another good drive) and 2. Above and beyond having some of the best speeds on the market, they also use 1GB of your RAM as a cache. Any repetitive I/O tasks will speed up incredibly from this.
There is a low risk of writes being made during a power loss being lost (as is true of anything, just a bit more so here), and if that worries at all, the Samsung 840 / 840 Pro's do not do this, and are also great drives (I've seen medical companies use them for clinical information if that's any endorsement)

Ultimately most recent drives released in the past year have decent Price/GB rates, high write speeds, and steady I/O over life span. I just recommend Samsung due to them controlling their factors of production, and have generally high standards for quality.


If anyone ever has any hardware questions, I'd be happy to share whatever expertise I have.

Aside from that, keep up the awesome work!
Super excited to hear the progress on the logistics/repair bots side. My friend and I were discussing game theory, and had come upon the topic that late game Logistics bots seemed to completely make other logistics systems irrelevant by trumping them all. By making a reasonable limiting factor, this means they can be an addition to existing systems, without being a full-on replacement for them. Really happy to see an elegant solution for a mechanics balancing "issue".
Most of your solutions have been wonderfully elegant, and I hope such wisdom continues, as it is going to make this game shine like a gem when it's finished.

Thank's for filling my free time with obsessive gamplay.

-Shriow

[RawCode]

also got OCZ, after half year performance on random reads dropped below 40 mbs (from 200), planning to switch into kingstone or intel drive.

[Shirow]

also got OCZ, after half year performance on random reads dropped below 40 mbs (from 200), planning to switch into kingstone or intel drive.

This all has to do with earlier generations of SSD's having performance degradation over the lifespan of the drive (Lots of technical reasons that are boring, if anyway cares I'll dig up some articles explaining)
The short of this being that when they switched to a new technology, they didn't properly plan the controllers for the long-term, which caused them to throttle themselves the more they are used (it's basically doing lots of garbage collection later on).

Typically performance degradations won't be obvious till the entire drive is written to at least once (due to garbage collection not being needed till then), and can be offset on bad drives by setting aside about 10% of the space (So if you have a 200GB drive, only using 180GB of it formatted. The drive automatically "write levels" across all it's cells, formatted or not. By having more cells free, it can be lazier with garbage collection without performance degradation).
Space reservation is common practice by many SSD manufacturers. Combining this with better controllers/firmware is how they've managed to bring stability up so much.

[Dysoch]

Ill be spending today to fix my computer to
Got a problem where it wont boot up. Its affected by my video cart. Normally when i pull it out and put it back in it works again, but not this time. I guess having one of the best setups available isnt enough

[Nova]

Just for information: OCZ has filed for bankruptcy.

[Psycho0124]

So I need to go and buy brand new ssd tomorrow (Any ideas what should I buy?) and spend the whole day by installing and configuring my computer, yay.

Your OCZ SSD, like most of the SSDs on the market, is a MLC (Multi-Layer Cell) drive.

SLC uses a single cell to store one bit of data. MLC memory is more complex and can interpret four digital states from a signal stored in a single cell. This makes it denser for a given area and so cheaper to produce, but it wears out faster.
.. an MLC cell is typically rated at 10,000 erase/write cycles, while an SLC cell might last 10 times that before failing.

source: http://www.computerweekly.com/feature/M ... ht-for-you

MLC SSDs are NOT ready for professional use. They just don't have the lifespan yet.
If you're on a budget, just go for a fast mechanical drive like the WD Raptor. I've had a pair in RAID-0 running flawlessly in my computers since '05 (I move them over every time I build a new rig and they just keep chugging!). If you really need the speed of an SSD, you should probably invest in a Single Layer Cell drive. They're smaller and more expensive but they don't suffer from short lifespans like the consumer MLC SSDs.

There are a few on Newegg:
http://www.newegg.com/Product/ProductLi ... isNodeId=1

and a few more on Amazon:
https://www.google.com/search?/s?sugexp ... rQGgjICADg

[SilverWarior]

@Kovarex
While I don't have direct expirience with SSD's I do have quite some expirience with regular HDD's.
One thing that you should try is to keep your SSD drive well coled when you will be copying the data from the old SSD to the new one. You wouldn't belive how many times well coled electronic board on normal HDD alowed me to make full backups of data when othervise the HDD kept shuting down due to faulty electronics. ANd since SSD completly rely on electronics this might help.
So I advise you to try coling the drive using aditional fan. It is best if the fan blows the air past the drive and not just directly into drive meanin if you have your drive mounted verticaly the air flow should also move vertically.

As for geting the data out of it, the best program I used so far is R-Studio (http://www.r-studio.com/).
With the difference of most other programs this one is a bit slow but this also means it causes less load to HDD/SSD which in turn decreases chance for failiure to happen.
It works with any HDD, SSD, Flash Cards and USB sticks. It also supports all latest Partitions.
The only problem might be the fact that this program is not free.

[kovarex]

Hello, thank you for your suggestions.
Finally, I bought the Samsung EVO 500GB, clonned the data from the old ssd in 30 minutes and I can resume the work at the exact point where I stopped and it seems to be stable.

[Psycho0124]

Ouch.. You went with a TLC (Three Layer Cell) design SSD?
That has an even lower lifespan than the flaky MLC drive you just had fail on you.

TLC- Three Layer Cell

Higher density
Lower endurance limit than MLC and SLC
TLC has slower read and write speeds than conventional MLC
5,000 program/erase cycles per cell
Best price point
A good fit for low-end basic products. Not suggested for critical or important applications at this time which require frequent updating of data.

http://www.centon.com/flash-products/chiptype

If you wore through a MLC drive in 9 months, you can expect only about 4.5 months out of your new drive, compared to about 8 years from a SLC drive and maybe 10 from a (well cooled) mechanical drive.

[Gammro]

Actually, the EVO series has some pretty impressive lifespans in terms of writes: http://www.anandtech.com/show/7173/sams ... s-tested/3

And while I couldn't get exact numbers on the vertex 3 lifespan, but those seem to have much less expected lifespan. So I'm pretty sure it's an overall improvement. Assumed they don't write more than 100GiB/day. In which case: Make sure you guys make (reliable) backups overnight.

It's not all about the base technology. It's a relatively new technique, and optimizations are constantly made.

[Shirow]

Ouch.. You went with a TLC (Three Layer Cell) design SSD?
That has an even lower lifespan than the flaky MLC drive you just had fail on you.

TLC- Three Layer Cell

Higher density
Lower endurance limit than MLC and SLC
TLC has slower read and write speeds than conventional MLC
5,000 program/erase cycles per cell
Best price point
A good fit for low-end basic products. Not suggested for critical or important applications at this time which require frequent updating of data.

http://www.centon.com/flash-products/chiptype

If you wore through a MLC drive in 9 months, you can expect only about 4.5 months out of your new drive, compared to about 8 years from a SLC drive and maybe 10 from a (well cooled) mechanical drive.

Yes, In the initial days (read: when OCZ was hitting the market) the SLC vs MLC distinction was incredibly important.
This was due to the fact no one in the industry had really worked with MLC and the impact of using that type of circuitry had. Most manufacturers flocked to it, due to it being the simplest and quickest way to increase data density / cost.

Two primary failures became very obvious (and I hate to say it, OCZ was the learning case for most other manufacturers. They were first to market for a lot, and held a lot of dominance at the beginning, they just never made a recovery off their early mistakes)
First is that SSD's inherently have a limited lifespan. This gets shorter with each process node making the chips smaller.
This was very bad when they started layering bit's on circuit boards due to the fact that a single bit write, would have to write the ENTIRE cell. If I remember correctly, SSD's have to read the data that's on it, erase it, and then write the state that it's on to do a single state. If you combine this with the fact you're only hitting part of the cell, you're running into an effect called Write Amplification where the amount of data being written is a multiple of your actual data.
That hits the second aspect, which is the impact on overall performance.
This has MASSIVE impact on overall lifespan and performance, which the first generation of SSD's were ill prepared for.

OCZ dealt with this by using a very heavy buffer, first introduced in their Vertex series, to try to "cope" with their performance decrease.
The vast majority of other controller's have put work into two coping areas.
1. Setting aside extra space so that when cell's starting failing, they'd hit reserve cell's, paired with "Write Leveling" where the controller write's to all cells once, and optimistically equally (causing the ENTIRE drive to need to be written x amount of times before any failure happens)
2. By making more intelligent controllers that buffer, compress, and predict data, to help more intelligently minimize the impact of messy data across the controller. This helps reduce write amplification, performance impact, and life durability.



tl;dr
MLC and TLC are not the nightmare's they once were.
Manufacturer's have gotten a lot more intelligent with design, and Samsung's one of the best (Since they make both their flash AND controllers, they know how to tweak them best).
Benchmarks and many critical reviews, as well as a growing enterprise adoption of SSD's have caused them to move from speculative/hobbiest, into a more mainstream position.

Finally,
Congrats on the 500GB Evo!
=D It's a hell of a drive, hope it serves you well.



Addendum : One quick note, when a SSD fails due to the flash medium wearing out, they become unable to write. This means that a "dead" SSD essentially becomes an CD-ROM. You can't write to it any more, but you've lost no data, as you can still read all of it. =) The performance degradation is a bit more relevant than "overall lifespan" as you're not fearing data-loss the same way you are when a spinning disc bites the dust (as those take a lot more to "read" from post-death)
Oh, almost forgot : SSD's aren't heat sensitive. While they shouldn't be boiled, making them "cool" for data recovery is based on spinning disc contraction. Good tip, wrong technology.

[RawCode]

if extreme amounts of IO operations needed, ramdrive + APS + backup SSD, it cost much, but provide RAM level performance and wont degrade over time.

[TOMII]

Yeah, I'm also in favor for RAM drive. Get a motherboard with support of 32-64Gb of RAM and set up a ramdisc on it.
Disadvantage is that you loose all the data which you were currently working with on power-loss so you will have to wait till they get copied once again on booting from the image. Ramdisc can work in few different ways, it can always start empty (fastest) or it can load an image at start (bigger image > longer boot up). About saving data, you can choose between manually/automatic/smart save back to image (on regular physical drive).
I'm not sure how much space do you use for compiling etc. But you wont find anything which will bring you more performance today.. The better MB(support of higher ram frequency) + faster FSB + good memory kits and you'r compiling time should decrease a lot
Speed example:

[kovarex]

Hello, thank you for the info.
The fault of the OCZ vertex was probably not lifetime problem as the S.M.A.R.T tool is indicating 100% health of the disk. If it was lifetime problem, the disk would slowly decrease it's size etc. But the problem was the random freezing, not sure why, but I guess it is some kind of deadlock in the firmware, some people had this problem with 3 days old ssd as well.
I can also see the statistics, I bought the disk just when i started working on Factorio, and It had 30T of write and 9T of read till this point.

About the ramdisk, I already tried to test the compilation on ramdisk some time ago, and there was no decrease of the time. The limiting factor of the compilation is the CPU at this point, not the disk.

[Shirow]

Hello, thank you for the info.
The fault of the OCZ vertex was probably not lifetime problem as the S.M.A.R.T tool is indicating 100% health of the disk. If it was lifetime problem, the disk would slowly decrease it's size etc. But the problem was the random freezing, not sure why, but I guess it is some kind of deadlock in the firmware, some people had this problem with 3 days old ssd as well.
I can also see the statistics, I bought the disk just when i started working on Factorio, and It had 30T of write and 9T of read till this point.

About the ramdisk, I already tried to test the compilation on ramdisk some time ago, and there was no decrease of the time. The limiting factor of the compilation is the CPU at this point, not the disk.

OCZ had poor controllers. This meant when it had to go back and "clean up" the cells before it could write to them, the controller would spend too much time trying to...essentially "Defrag" the incoming data, and where it needed to go, causing microstutters. Combine this with lots of random writes that your OS naturally does, and some lower cache's, and you get their entire drive freezing.

Also, It's official, all OCZ assets are being bought by Toshiba.