477 lines
14 KiB
JSON
477 lines
14 KiB
JSON
[
|
|
{
|
|
"start": 317.39999998,
|
|
"end": 322.76,
|
|
"text": "And then any server was only working off of its own local database."
|
|
},
|
|
{
|
|
"start": 323.0,
|
|
"end": 335.685,
|
|
"text": "And then every time you made a call, for example, create an account on one of the servers. If it was the primary server, it will write into its own database, then use the RMI call to go to to the servlets to"
|
|
},
|
|
{
|
|
"start": 336.005001,
|
|
"end": 340.485,
|
|
"text": "go to the other server and tell it now write this in your own database,"
|
|
},
|
|
{
|
|
"start": 340.645,
|
|
"end": 348.489998,
|
|
"text": "receive back a I got it, and then only then mark the adding as successful in the primary database."
|
|
},
|
|
{
|
|
"start": 348.81,
|
|
"end": 351.37,
|
|
"text": "If the writing was in the secondary database,"
|
|
},
|
|
{
|
|
"start": 351.85,
|
|
"end": 352.809998,
|
|
"text": "it would"
|
|
},
|
|
{
|
|
"start": 353.77,
|
|
"end": 359.815,
|
|
"text": "write it to the secondary, sent to the primary, primary the acknowledges, and then the primary sends back again"
|
|
},
|
|
{
|
|
"start": 359.815,
|
|
"end": 364.855,
|
|
"text": "as a distribution because it may have other servers other secondaries."
|
|
},
|
|
{
|
|
"start": 365.415,
|
|
"end": 367.655,
|
|
"text": "So this was for any update in the database."
|
|
},
|
|
{
|
|
"start": 368.55,
|
|
"end": 373.03,
|
|
"text": "Tracking table was only on the primary. As we all know, server locks were always local."
|
|
},
|
|
{
|
|
"start": 373.59000000000003,
|
|
"end": 381.59000000000003,
|
|
"text": "And when a server job had to be done, the primary was picking up who to do it based on what it told the other server is doing."
|
|
},
|
|
{
|
|
"start": 382.255005,
|
|
"end": 389.775,
|
|
"text": "That was the model. It was application level only. The two MariaDB's didn't know about each other in any any way or form,"
|
|
},
|
|
{
|
|
"start": 390.33501,
|
|
"end": 393.53499999999997,
|
|
"text": "and each server was working individual individually"
|
|
},
|
|
{
|
|
"start": 393.53499999999997,
|
|
"end": 394.975006,
|
|
"text": "with its own database."
|
|
},
|
|
{
|
|
"start": 395.98,
|
|
"end": 397.340004,
|
|
"text": "With Postgre,"
|
|
},
|
|
{
|
|
"start": 397.420006,
|
|
"end": 400.380005,
|
|
"text": "we're basically changing the model completely."
|
|
},
|
|
{
|
|
"start": 401.100006,
|
|
"end": 402.22,
|
|
"text": "First of all,"
|
|
},
|
|
{
|
|
"start": 402.70000500000003,
|
|
"end": 410.060005,
|
|
"text": "during a proper operation of the cluster, all the cluster nodes will work off the primary database."
|
|
},
|
|
{
|
|
"start": 410.925,
|
|
"end": 411.565,
|
|
"text": "So"
|
|
},
|
|
{
|
|
"start": 412.525,
|
|
"end": 420.045,
|
|
"text": "both server one and server two will be connected live to the primary database for all of this configuration"
|
|
},
|
|
{
|
|
"start": 421.485,
|
|
"end": 430.17,
|
|
"text": "of anything in the database besides the server logs. So you don't have the primary secondary concept anymore. It's just a primary secondary database."
|
|
},
|
|
{
|
|
"start": 431.369995,
|
|
"end": 433.85,
|
|
"text": "That's one of the things. The replication,"
|
|
},
|
|
{
|
|
"start": 433.85,
|
|
"end": 448.565,
|
|
"text": "how the secondary is kept in in on on track is not to the application anymore. Instead, we we set up replications on the database level. So we have a new page. Let me share my screen because I was just looking at that a few seconds ago."
|
|
},
|
|
{
|
|
"start": 449.445,
|
|
"end": 451.365,
|
|
"text": "Let me know when you see my screen."
|
|
},
|
|
{
|
|
"start": 451.685,
|
|
"end": 454.8,
|
|
"text": "I can see it. So that is the September build,"
|
|
},
|
|
{
|
|
"start": 454.96000000000004,
|
|
"end": 465.12001,
|
|
"text": "and this is how it looks. It looks very familiar. Right? Because it's basically just says Postgre instead of Maria. But now there is the replication settings here at the end. And if you have a cluster,"
|
|
},
|
|
{
|
|
"start": 465.36,
|
|
"end": 471.17502,
|
|
"text": "which I don't have, it's a single one, you can start the database replication, set up a password,"
|
|
},
|
|
{
|
|
"start": 471.33500000000004,
|
|
"end": 472.53499999999997,
|
|
"text": "and set up here"
|
|
},
|
|
{
|
|
"start": 473.095,
|
|
"end": 476.05501000000004,
|
|
"text": "what parts of the database will be replicated."
|
|
},
|
|
{
|
|
"start": 476.13500999999997,
|
|
"end": 482.2,
|
|
"text": "If there is a cluster, this will be filled in with the notes and so on. And this is how the replication happens."
|
|
},
|
|
{
|
|
"start": 482.51999,
|
|
"end": 489.48,
|
|
"text": "So that in the live operation, what happens is that server one and server two talk to database one only."
|
|
},
|
|
{
|
|
"start": 490.51999,
|
|
"end": 496.095,
|
|
"text": "It behind the scenes, except for the locks which stay local in the local database. So I'll just"
|
|
},
|
|
{
|
|
"start": 496.975,
|
|
"end": 503.21501,
|
|
"text": "the ST server lock, the whole space just basically doesn't replicate, although it actually can be set up to replicate if needed."
|
|
},
|
|
{
|
|
"start": 504.735,
|
|
"end": 505.37502,
|
|
"text": "When"
|
|
},
|
|
{
|
|
"start": 505.855,
|
|
"end": 509.9,
|
|
"text": "something changes in the database, it's the Postgre application,"
|
|
},
|
|
{
|
|
"start": 509.9,
|
|
"end": 513.74,
|
|
"text": "the internal Postgre application that we had weaponized, basically,"
|
|
},
|
|
{
|
|
"start": 513.9,
|
|
"end": 519.74,
|
|
"text": "to use to replicate the secondary database. So the secondary database is essentially just sitting there,"
|
|
},
|
|
{
|
|
"start": 520.06,
|
|
"end": 524.925,
|
|
"text": "serving only locks when someone touches something. The sec the"
|
|
},
|
|
{
|
|
"start": 525.645,
|
|
"end": 529.88501,
|
|
"text": "because of that, the secondary server actually have connection to both databases,"
|
|
},
|
|
{
|
|
"start": 529.965,
|
|
"end": 532.205,
|
|
"text": "but the live connections is only"
|
|
},
|
|
{
|
|
"start": 532.685,
|
|
"end": 533.965,
|
|
"text": "in the primary,"
|
|
},
|
|
{
|
|
"start": 534.58,
|
|
"end": 539.22,
|
|
"text": "which gives you on top of everything the queue management of the enterprise cluster now."
|
|
},
|
|
{
|
|
"start": 539.78,
|
|
"end": 541.46,
|
|
"text": "Because in the old world,"
|
|
},
|
|
{
|
|
"start": 542.57999,
|
|
"end": 553.265,
|
|
"text": "the queue was managed by the primary, while now each of the nodes actually can pick its own jobs again from the database because we remove the old way to distribute jobs."
|
|
},
|
|
{
|
|
"start": 553.825,
|
|
"end": 554.465,
|
|
"text": "So"
|
|
},
|
|
{
|
|
"start": 554.625,
|
|
"end": 559.825,
|
|
"text": "in some ways, you can think that as a scaled down version of the enterprise cluster,"
|
|
},
|
|
{
|
|
"start": 559.825,
|
|
"end": 561.265,
|
|
"text": "but with an embedded database."
|
|
},
|
|
{
|
|
"start": 563.64,
|
|
"end": 569.24,
|
|
"text": "And that's pretty much what it does. Anything we had been explaining how the enterprise cluster is better for"
|
|
},
|
|
{
|
|
"start": 570.04,
|
|
"end": 573.72,
|
|
"text": "distributing jobs now comes into the standard cluster as well."
|
|
},
|
|
{
|
|
"start": 574.28,
|
|
"end": 579.07498,
|
|
"text": "Because of that, there is additional requirement for more users. This is for the replication."
|
|
},
|
|
{
|
|
"start": 579.5550000000001,
|
|
"end": 582.915,
|
|
"text": "There is additional requirement for more ports to be open"
|
|
},
|
|
{
|
|
"start": 583.155,
|
|
"end": 592.91498,
|
|
"text": "so that the databases can talk to each other and so that the secondaries can reach the proper database. In a case of a fail off, if something happens on the primary"
|
|
},
|
|
{
|
|
"start": 592.84998,
|
|
"end": 595.4100000000001,
|
|
"text": "and another server becomes a primary,"
|
|
},
|
|
{
|
|
"start": 595.8899799999999,
|
|
"end": 603.25,
|
|
"text": "the database on the current primary becomes the primary for everyone. So everyone in the cluster will join to the other primary,"
|
|
},
|
|
{
|
|
"start": 603.81,
|
|
"end": 607.16998,
|
|
"text": "which also means that you don't have sync button anymore"
|
|
},
|
|
{
|
|
"start": 607.585,
|
|
"end": 612.1450199999999,
|
|
"text": "because there is no manual synchronization needed because you have only one valid database."
|
|
},
|
|
{
|
|
"start": 612.9449999999999,
|
|
"end": 614.1450199999999,
|
|
"text": "So the automatic"
|
|
},
|
|
{
|
|
"start": 614.385,
|
|
"end": 621.33002,
|
|
"text": "when the automatic failover happens, everyone will just rejoins to the proper database, and that's it. You don't need to do anything."
|
|
},
|
|
{
|
|
"start": 621.65002,
|
|
"end": 626.05002,
|
|
"text": "If the what old primary shows up and becomes a primary gate,"
|
|
},
|
|
{
|
|
"start": 627.4100000000001,
|
|
"end": 630.21002,
|
|
"text": "it will pick up again. So it's just"
|
|
},
|
|
{
|
|
"start": 630.69,
|
|
"end": 636.4449999999999,
|
|
"text": "a totally different model. It will take a little while to get used to it, by the way. But I'm"
|
|
},
|
|
{
|
|
"start": 636.685,
|
|
"end": 641.005,
|
|
"text": "I'm pretty sure that everyone is happy we don't have a manual sync button anymore."
|
|
},
|
|
{
|
|
"start": 641.8050000000001,
|
|
"end": 642.525,
|
|
"text": "Mhmm."
|
|
},
|
|
{
|
|
"start": 643.565,
|
|
"end": 647.565,
|
|
"text": "Yeah. It sounds like it will solve our in progress transactions"
|
|
},
|
|
{
|
|
"start": 647.8050000000001,
|
|
"end": 655.6700000000001,
|
|
"text": "that they keep floating. It's like the only way to get rid of them is to delete the subscription that where it's happening and then recreate it."
|
|
},
|
|
{
|
|
"start": 655.99002,
|
|
"end": 662.63498,
|
|
"text": "And then, you know, then it goes away, but then it floats to another one. So we've we've been having that that problem. And"
|
|
},
|
|
{
|
|
"start": 663.435,
|
|
"end": 664.5550000000001,
|
|
"text": "Yeah. Go ahead."
|
|
},
|
|
{
|
|
"start": 665.115,
|
|
"end": 675.3549800000001,
|
|
"text": "Yeah. In in the you know, it sounds like it's just minor and annoying, but, actually, at some point, it can stop the primary from actually doing anything, delegating anything."
|
|
},
|
|
{
|
|
"start": 675.76,
|
|
"end": 684.0,
|
|
"text": "Yeah. Yeah. Yeah. When the primary is too busy with stuck jobs, it doesn't have any power to send anything to anyone else. In the new model,"
|
|
},
|
|
{
|
|
"start": 684.24,
|
|
"end": 690.08002,
|
|
"text": "there is no primary dispatcher anymore. So because everyone is basically on their own, well, not exactly,"
|
|
},
|
|
{
|
|
"start": 690.875,
|
|
"end": 697.595,
|
|
"text": "but it also is a lot less likely to kill your secondary by sending it too many heavy jobs while it's busy with app. So"
|
|
},
|
|
{
|
|
"start": 698.155,
|
|
"end": 707.1800000000001,
|
|
"text": "one of the problems with all the old standard cluster is that the primary doesn't really know how busy the secondary is. So if the secondary is on the brink of failing,"
|
|
},
|
|
{
|
|
"start": 707.81998,
|
|
"end": 727.225,
|
|
"text": "in an enterprise cluster, the server knows not to pick up more jobs. Right? So it will just stop picking jobs for a while until it can self heal. In the standard cluster, this mechanism didn't work because there was no way for the primary to know that the secondary needs to heal. And when the primary was trying to heal, nothing was working. It was everything was blocked"
|
|
},
|
|
{
|
|
"start": 727.5450000000001,
|
|
"end": 729.625,
|
|
"text": "because there was no dispatcher."
|
|
},
|
|
{
|
|
"start": 729.625,
|
|
"end": 730.505,
|
|
"text": "Now that changes."
|
|
},
|
|
{
|
|
"start": 731.40002,
|
|
"end": 740.76,
|
|
"text": "Part of the reason we did that is because we wanted to eliminate one of the databases. Part of it is we're using a very old library for the dispatcher model that we had to get rid of,"
|
|
},
|
|
{
|
|
"start": 741.08002,
|
|
"end": 745.1600000000001,
|
|
"text": "and there was no update for it. And part of it is we know the other model works."
|
|
},
|
|
{
|
|
"start": 746.0349699999999,
|
|
"end": 759.63498,
|
|
"text": "And here is where I will actually warn people something because I don't know because now both servers are going to go against the primary database, this primary database will actually need more power than the old MariaDB database"
|
|
},
|
|
{
|
|
"start": 760.24,
|
|
"end": 763.2,
|
|
"text": "because you have two processing engine connecting to it."
|
|
},
|
|
{
|
|
"start": 764.24,
|
|
"end": 764.80002,
|
|
"text": "So"
|
|
},
|
|
{
|
|
"start": 765.28,
|
|
"end": 773.6800000000001,
|
|
"text": "if someone is planning something and you are low on resources on your primary box, that's the time to start thinking about a little bit more memory and CPU"
|
|
},
|
|
{
|
|
"start": 773.94498,
|
|
"end": 777.3050000000001,
|
|
"text": "just to assist your database and IO and so on,"
|
|
},
|
|
{
|
|
"start": 777.7049999999999,
|
|
"end": 778.82498,
|
|
"text": "just as heads up."
|
|
},
|
|
{
|
|
"start": 779.145,
|
|
"end": 784.425,
|
|
"text": "So Is that part of the documentation, you know, installations? Like, well, you know,"
|
|
},
|
|
{
|
|
"start": 786.185,
|
|
"end": 789.2700199999999,
|
|
"text": "per files or per per load,"
|
|
},
|
|
{
|
|
"start": 789.2700199999999,
|
|
"end": 791.35,
|
|
"text": "you would need this many CPUs,"
|
|
},
|
|
{
|
|
"start": 791.51,
|
|
"end": 792.63004,
|
|
"text": "this much RAM?"
|
|
},
|
|
{
|
|
"start": 793.03,
|
|
"end": 803.43002,
|
|
"text": "Well, yes and no. We actually changed the install guide to have a different minimum than it used to be. So we actually updated the minimums to be to something that resembles a production server."
|
|
},
|
|
{
|
|
"start": 804.125,
|
|
"end": 805.8050000000001,
|
|
"text": "So if you follow those,"
|
|
},
|
|
{
|
|
"start": 806.44498,
|
|
"end": 816.125,
|
|
"text": "you are more likely to succeed. However, if you are an existing customer and you are on the brink of overflowing your servers and you are riding very close to your capacity,"
|
|
},
|
|
{
|
|
"start": 817.41998,
|
|
"end": 825.58002,
|
|
"text": "you probably should update your primary a little bit or your servers a little bit before going into the new cluster just to be on the safe side."
|
|
},
|
|
{
|
|
"start": 826.38,
|
|
"end": 827.5,
|
|
"text": "It drops"
|
|
},
|
|
{
|
|
"start": 827.5,
|
|
"end": 831.66,
|
|
"text": "some of the load that was part of the dispatcher model, but you still have"
|
|
},
|
|
{
|
|
"start": 831.905,
|
|
"end": 840.30505,
|
|
"text": "two servers working on the same database. So, essentially, your primary box now will run an SD server and the whole database for both servers."
|
|
},
|
|
{
|
|
"start": 841.265,
|
|
"end": 847.18506,
|
|
"text": "So You recommend that the Postgres SQL live on the primary server, or can it be"
|
|
}
|
|
] |