| Krinkle | |
| Oct 3 2013, 8:15 PM |
| F12690: req-vebase_nocrash.png | |
| Nov 22 2014, 2:38 AM |
| F12692: req-vecore_nocrash.png | |
| Nov 22 2014, 2:38 AM |
| F12691: req-vecore_crash.png | |
| Nov 22 2014, 2:38 AM |
load.php request for ve.base (when it crashes)
At this time, on most Wikipedias that have just been switched to 1.22wmf19, VisualEditor is fatally broken.
It cannot be consistently reproduced, but trying 3 or 4 times will hit the exception.
Version: unspecified
Severity: critical
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=51766
Attached:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Krinkle | T56945 ResourceLoader: Cache issues (tracking) | |||
| Resolved | Krinkle | T56935 VisualEditor: Editor does not load at all (ve.ui.TargetToolbar is undefined) |
Created attachment 13434
load.php request for ve.base (when it does not crash)
Attached:
Created attachment 13436
load.php request for ve.core (when it does not crash)
Attached:
The two ve.base request are (naturally) from different cache servers, but the Content-Length is identical, and the Age is close enough (sent out at different times, and my request is at a few seconds later as well).
However the ve.core request is significantly different (eventhough we're giving the exact same url with the same timestamp from the startup module).
I think this is a case where it once again shows how conceptually flawed our deployment system is (swapping out files and directory inside /a/common while apache is still fully serving and pooled) – or, how we did not take this into account in the design of ResourceLoader (take your pick).
As a result I think one of the following 2 scenarios happened:
Scenario A:
Imagine module bar depends on the (new version of) module foo. From this point on, users hitting srv100 for the module=foo request will keep getting an old version and results in broken scripts in unpredictable ways.
Scenario B:
In this scenario a dependency between different requests/modules doesn't even matter because in our deployment system a server can even be in mixed state within itself (as opposed to a mixed state across the datacenter, as scenario A).
Again, it is cached under the new timestamp.
Contrary to what one might think, both scenarios do not automatically resolve themselves the next 5-minute window of the startup module.
The next time the startup module is generated, 5 minutes later, the max() timestamp of a module will still be the same.
Touching startup.js won't help either.
You'd have to artificially touch the individual module that got corrupted in cache and sync it and hope the same race condition won't happen again.
Fixed by touching the JS file in production.
And yeah, this is another artefact of 43805, I think. :-(
