# Transcript: 1045479741
# URL: https://vimeo.com/1045479741
# Duration: 6025s (100.4 min)

[0:02] Okay. Alright.
[0:07] Well, good morning, good afternoon.
[0:10] Thank you all for joining today.
[0:13] And before we get into
[0:15] the ask Annie this this morning for us, just wanna quickly go over a couple of things
[0:21] to bring your attention to.
[0:24] And
[0:25] first of all, as
[0:27] you all know that have been joining these for quite some time or or new to the user group, this webinar is being recorded. So we will send the recording and presentation after
[0:38] the the session so you can look for that. And, of course, whenever you have any questions, feel free to unmute yourself or throw them in the chat, and Annie will be glad to address those
[0:50] either on the call or after the call. So
[0:53] don't hesitate to ask any questions that you may have.
[0:58] Another thing,
[0:59] our Axway Summits for twenty twenty five
[1:02] are coming up, and we're excited to present those to bring your attention to that. The North America one will be in Austin, Texas, March
[1:11] 2526.
[1:12] So if you are
[1:14] wanting to go to that, please do register.
[1:17] And again, the links are here, and you'll be able to click on those in the presentation. But we're very excited about that.
[1:24] These are great opportunities for you to have
[1:29] interact with peers and the Axway team
[1:33] and just hear real world case studies,
[1:36] attend informative sessions.
[1:39] For those of you that attended summits in the past, they really are excellent opportunities to really
[1:45] interact with everybody and get some great information.
[1:48] So if you need any more information on that, feel free to go to the link and it will
[1:54] provide the agenda and all the
[1:57] details for that. And for those of you that
[2:00] would love to go to Barcelona, Spain,
[2:02] we are conducting that one June.
[2:05] So
[2:06] we'd love to see you at either one of those, and
[2:10] let us know if you have any more questions. But
[2:13] wanted to let you know about that. Another thing, we do customer hosted
[2:18] user group sessions.
[2:20] Costco, Dun and Bradstreet, Lockheed Martin, and Soper Seria. We've had these at the customer sites.
[2:27] And all you have to do is, if you would like to do one, is to
[2:32] provide the meeting room and we take it from there. We provide a light continental breakfast and a lunch, and it's usually about nine to four, somewhere in there, nine to three, nine to four. So nice all day session,
[2:44] and Annie
[2:46] leads it. And then a few other X-ray team members are usually there on-site to present.
[2:51] And
[2:52] really great opportunity for you to not have to travel.
[2:56] And you and your team members can can be there and just avoid all that hassle of traveling. So
[3:03] if you
[3:05] think your company would be interested in in hosting one of these, and, again, just hosting by providing the meeting room, and then
[3:12] please let me know,
[3:14] and we'll be glad to connect on that. So
[3:18] wanted to bring your attention to that. And before Annie jumps in here,
[3:23] we have a peer to peer program that is fairly new,
[3:28] and it's just a great kind of networking
[3:31] connection
[3:32] where we we connect you with people that are
[3:37] usually in your
[3:39] your
[3:41] time zone and
[3:43] allow you to just connect with people that you wanna just talk to and and
[3:48] have conversations
[3:49] with with, you know, discussions and questions that you have and just bringing your kind of where the matchmaker for you and stuff. And with this has been getting gaining some great
[4:02] it's been it's really been well received.
[4:05] So we're excited about it, and a lot of our customers are really enjoying that. So if you're interested,
[4:11] all you have to do is just email us at community dot x way dot com and let us know that you're interested, and we'll
[4:17] get you connected with
[4:19] people that share your experiences and
[4:22] really could benefit you two can you can benefit from each other.
[4:28] And without further ado,
[4:30] ask Annie. We'll now begin with Annie Otova.
[4:34] Good morning. Good afternoon, everyone.
[4:37] Welcome to our first session for the year. I don't know what happened. They erode again.
[4:43] So,
[4:45] a lit a couple of housekeeping items before we start.
[4:48] If you have a question, as Lucy mentioned, unmute or put it in the chat. I'm monitoring the chat as well. We'll try to keep something some some things in order.
[4:58] If we're in the middle of a question and yours is related, feel free to immediately chime in. If it's not related, if you don't mind just raising your hand so I know you're waiting for a question so that we can try
[5:11] to do some order and finish one conversation before move to another, although
[5:17] we can always return to the other one. With that being said,
[5:22] that's about it on the house, housekeeping items, and of course, if you are here for the first time, this is all about ST,
[5:30] so
[5:31] any questions goes. A checkbox question or architecture question, it doesn't really matter. If it is about ST in some way or form,
[5:40] please ask it. If I don't know the answer, I'll tell you I don't know. If we can figure it out from the server itself, I have a live server,
[5:49] we'll do that. If not, I'll get back to you later.
[5:52] And with that, we actually have a quest someone actually proactively sent me a question so they get priority.
[5:58] I also say saw one in the chat already, so we already have two questions in. So let's get started. Lucy, can you just push to the next slide for me?
[6:07] I think I saw Hans in.
[6:09] Hey, Hans.
[6:11] Yeah. Hi, everybody.
[6:13] Hello. Good afternoon. Good morning.
[6:15] Yeah.
[6:16] This is my question.
[6:18] Yeah. Hans had been very good in the last couple of sessions sending his questions a day early, which helps with me actually getting my head straight on some of them.
[6:28] It's not a requirement, but it helps organize things and also help me bring whatever is to assign it. So just mentioning it. It's again,
[6:37] it's okay to just show up and ask a question, but if you can if you send it earlier, it'll be on the screen and going first. So
[6:44] so, Hans,
[6:46] so
[6:47] I think this is also a continuation of a discussion we had last year with you. Right? Yeah. Yeah. Yeah. So you anyone
[6:54] so I'll just summarize. I put the whole question as is simply because I wanted to cover a couple of points exactly as they were
[7:02] just because there are a couple of warnings there. So in summary,
[7:06] you need to update your OS
[7:09] from
[7:10] Red Hat eight to Red Hat nine,
[7:13] which will require new servers being installed as opposed to in place upgrade, which will mean and the question is how to move your SD with you to the new servers without disrupting traffic.
[7:25] And that's Yes. The migration question.
[7:28] So
[7:30] your so and what you're asking is, you do the easy thing?
[7:35] Stop the servers, tarball them, enter on the other end, and call it a day.
[7:40] And here, the answer is a little
[7:45] there are two answers to that. From the technical side, this will most likely work.
[7:51] However, it will not be supported.
[7:54] And here is where with ST, there are a couple of areas like that where there are things that we kinda sort of expect to work. However,
[8:03] they have never been tested and not officially documented.
[8:08] So Yep. Even though in theory,
[8:11] this should work,
[8:12] in practice, if you try to do that and you meet some weird problems later,
[8:17] support will
[8:19] bulk on you because it's not supported,
[8:23] unfortunately.
[8:24] So Yeah.
[8:25] Officially,
[8:26] the official line is that every server that you run should start with an installer.
[8:33] You cannot just start both from somewhere else. So now between eight and nine, because both are Red Hat, I don't think there will be a binary changes.
[8:42] However,
[8:43] when we install, there is a couple of
[8:47] native libraries that Java is carrying with itself,
[8:50] and I don't know if we had done a study to verify that those actually don't need a little bit of a replacement.
[8:57] So
[8:59] I wouldn't do it the way you're listing. I wouldn't do a,
[9:05] So if you want to do that for a lower environment
[9:09] that
[9:11] you
[9:12] want to
[9:13] just you know, that you can throw away if something goes wrong,
[9:17] be my guest, but I wouldn't do that for production.
[9:20] What I will do for production
[9:22] will be to play with the disaster recovery mechanism.
[9:25] It will require an additional database so you can install the new cluster against it.
[9:31] But
[9:32] once everything is done, you can repoint them to the live database. That way, you'll still have everything. What you will lose will be the log files in the file system, and those you can move manually.
[9:43] Those, I would say, move them.
[9:45] If you care about matter.
[9:47] Have it in Splunk. It doesn't matter. So Yeah. And that's why I said that's the only thing you'll lose.
[9:52] So in so if
[9:55] you want, open a ticket with support with this plan and see if they will bless it. If our support says okay, I'm not going to to disagree.
[10:03] Right? Yeah. Yeah. I've done things like that on not on lower environment where I know I can blow them away if something goes wrong.
[10:11] One thing to mention here is
[10:14] when you say file drive home, you mean the SD itself. If your installed folder is in a different place, and this is actually that's why I wanted to put it here. When you're doing the backups
[10:25] so this backup is supportable on the same server only, technically speaking.
[10:29] And and you'll hear a lot of technically speaking because there is a difference between what will work and what will be supported. Right? So Yeah. Don't forget the installer folder. It may not be under file drive home
[10:41] Yep. Because you cannot update without it.
[10:45] But that's about it. For the edges,
[10:48] honestly,
[10:50] I don't
[10:51] even know that you won't even try something like that. It's a lot faster to just install new edges and import the three certificates you need.
[10:59] Yeah. Yeah. Yeah. Just saying. And that's another thing that I wanted to bring up is whenever you were doing any planning about backups and so on,
[11:08] especially
[11:09] if you have your, server logs already rescued somewhere else like Splunk or something from the edges, recovering an edge is
[11:17] a better idea time wise and effort wise.
[11:21] If you if it will take you more than fifteen minutes to recover, you'll you'll have reinstalled and reconfigure to new edge by then. Just saying. Yeah. Right?
[11:29] Yeah. It's easier to edge. It's easier. Yeah. And
[11:33] this will not be zero downtime per se when
[11:37] you do it with the Doctor mechanism. However, your downtime will be just this moment
[11:43] moment when you switch down the servers and switch the new ones in.
[11:46] So it's a very quick switch because you mostly
[11:50] configure the servers on the other end. As part of the gear process, you replace the IDs of the new servers to mimic the IDs of the old servers.
[11:59] So the moment when the new server slide into the place of the old servers, they'll essentially be the same servers for the database. So everything will continue from where it was. So there is a very small restart over there. So I
[12:14] I know what you're trying to achieve, as I said, that's why I put it in here.
[12:19] But it's
[12:21] it's weird. And if you look at the chat window, Cam just did another upgrade like that
[12:28] couple of weeks ago.
[12:30] And
[12:31] ping her, talk to her.
[12:34] She had been doing ST longer than most of the people I know.
[12:38] Not as long as me, but thereabouts.
[12:41] Yeah. But
[12:43] so but again, one of the things with ST is there is a lot of things that technically will work.
[12:49] But either we haven't done enough studies
[12:52] or there is a weird
[12:54] corner case that we hit with another customer. This is why we don't qualify them, and we don't always have a documentation about it.
[13:02] Right? Mhmm. And that's why whenever you are trying something as unorthodox
[13:06] something unorthodox going talking to the support
[13:09] is where you go. Because if they know that we had tested it exclusively
[13:14] and just there isn't time to finish testing, they might place it as is.
[13:19] But I would honestly not do that.
[13:23] Simply be not not do your plan, not because it won't work on the surface.
[13:28] What I'm worried is three updates down the road
[13:31] where it turns out something should have been done a little differently on the new OS, and we finally hit it. Mhmm. Right?
[13:38] So, you know, it might work for the next six months, and then six months down the road, you do a Gotham variety update,
[13:46] and everything falls apart.
[13:48] And you and support spend the next two weeks trying to to figure out what happens, and it turns out it's because the installer should have done something, but because you never run it, it didn't. Right?
[13:59] So Yeah. It's one of those risks. Again, low environment that you can reinstall without troubles, be my guest. I would never do it any other way unless I'm doing it for testing anyway.
[14:10] Yeah. Yeah. Yeah. Of course. We have a two test environment, and
[14:14] I would do it before
[14:16] I go to production.
[14:17] That's that's good for me. But what I'm saying is don't
[14:21] but don't rely on, oh, it worked, so let me make let me do it the same way. It's
[14:27] we haven't done the studies to make sure that it will. So just just
[14:32] it's one
[14:33] of those things.
[14:35] I would love to be able to tell you, yeah, go ahead and do that because it's obviously the easiest way to do things. Right? We're not there yet.
[14:44] Maybe one day.
[14:46] Yeah.
[14:47] That's why I asked, is it so simple?
[14:50] So yeah.
[14:52] There were days when it was so simple. Before we have a database and so on. These days, and especially with environment like yours with enterprise cluster only and so on, the last thing you want is to shoot yourself in the foot downstream.
[15:07] 90%
[15:08] of the issues like that don't propagate during the updated cell. Don't propagate in the first months. It's somewhere downstream when we rely on something being there because you're on this OS.
[15:20] Mhmm. But it's not there because you just carried it over.
[15:24] So,
[15:26] you know, and that's the other thing. I would also talk to your Linux admin. Because
[15:32] if the
[15:33] if there was an update from eight to nine in place, even if they chose not to do it,
[15:40] it's a much, much bigger chance that everything will actually work on the other end because you don't need to reinstall after an update if it is in place.
[15:48] Yeah. Yeah.
[15:49] Yeah.
[15:51] And as Chris mentioned, there is also when you whenever you are doing that, if are you going to reuse the IPs, or are you going to have new IPs for the servers?
[16:01] I
[16:02] like to to use the same IP, but but if if if they
[16:07] if that means that that's opportunity to
[16:10] to
[16:11] build a new VM to move with new IP addresses and all the stuff, and then I have to install the
[16:18] ST, for example.
[16:20] Mhmm.
[16:21] But,
[16:22] I like
[16:24] because, of course, if I have new IP, I have to reconfigure
[16:29] Yeah. Some things in the cluster.
[16:31] And not just the cluster. If you reconfigure the IPs on the server, don't forget to reconfigure the edges to allow the new Yeah. Servers.
[16:39] It's not remember, you have to so on the edges, there is the allowed servers list, and that's the one that is important because this is where your SOX proxy
[16:48] enablement is. Because we don't you know, our SOX proxy is not authenticated.
[16:53] Yeah. We just do a white listing.
[16:55] So and that's one of the major problems I'm seeing.
[16:58] Other than that, history itself will make it work, But people tend to forget this allowed servers list up there. And if we're building new edges with new IPs, don't forget the zones down in the server.
[17:12] And and
[17:13] that's the other thing.
[17:14] Because the
[17:16] IP
[17:17] so because
[17:18] what release are you on?
[17:22] On the actually, the on the September
[17:25] release of
[17:27] '24.
[17:28] Okay. So this is already the new the new database?
[17:32] Yes. Yes. So so on the edges, especially with the new date so the new database, Postgre,
[17:39] has a additional additional
[17:41] place where the IP of the current box is actually in the configuration that didn't used to exist with MySQL.
[17:49] Yeah. So this will come and guide you. So especially for the edges,
[17:53] you know, export the five parameters you are using, export and or do something with the API if you want to, you know, it's straightforward. Get the couple of certificates and just reconfigure the edges from scratch.
[18:06] And if you're going to change the IPs,
[18:08] you know, build the whole thing on the site
[18:11] with a separate database. I know you'll need a database for a couple of hour, for a couple of days, or a couple of weeks. I know Oracle is not cheap, but, you know,
[18:21] but configure everything.
[18:24] But don't forget that the moment when you swing to the live database, you are back into the live database. So if your edges have new IPs,
[18:31] before you turn them on, you'll need to change those values in the zones.
[18:36] Okay. Yep. Yep. Yep. I see. You know, it's it's it's just about orchestration.
[18:41] It's pretty straightforward. That's I'm that's why I'm saying design
[18:44] that based on the g r model.
[18:47] The g r model look very cumbersome,
[18:49] but if you look at it, it's perfect for this kind of stuff. Because it will allow you to get everything exactly as is and the server are supposed to be replicas.
[18:59] Yep.
[19:01] That's fine. Yeah.
[19:04] So, Chris, you don't need to so Chris has an additional question. How would you update the database for the enterprise cluster for the new IPs?
[19:11] Well, this is where the beauty of the whole model is because we have the override file, and those can so if you're asking for the IP of the cluster itself,
[19:21] if you look at the Doctor model description, it actually shows you how to update and to, how to update the IP
[19:28] during the startup of the Doctor servers. You use the same thing. There is an override file with specified IP. For the zones,
[19:35] you can either do it after you start everything,
[19:39] or you can do it just before you shut down the old cluster
[19:42] either way or use the API or after the first admin is up.
[19:47] And just to highlight, we have new documentation about Doctor.
[19:51] Because they were working on CDU and Doctor together
[19:55] and documentation for that, we actually rewrote the whole disaster recovery part of the admin guide for December.
[20:03] So but and, Hans, even though you are on
[20:07] September,
[20:09] there is nothing in this guide that will not work for you.
[20:13] So if you want to go to the documentation page and look at the disaster recovery plan, it had been it had it a lot of new clarifications.
[20:22] And, again, as you know, the live documentation is for the current version.
[20:27] The offline one, when you click on the button, is for your current version, and you shouldn't should usually should be
[20:33] either comparing them or staying with that.
[20:36] Mhmm. But the one for the Doctor that got rewritten is just a rewrite to clarify things to make it clearer,
[20:43] and there is nothing new in there. So anything in there will gonna work for September. It will probably work for anything for the last four years
[20:50] anyway. Okay. Yeah. Yeah. Okay. So just
[20:54] just mentioning.
[20:55] Okay.
[20:59] Thank you very much for your explanation.
[21:01] Okay. Well, good luck with your update. And as I said, talk to you I suspect I'll talk to you next month again anyway.
[21:09] So Yeah. Yeah. Yeah. Yeah. Keep me keep us updated. And if you, again, if you really, really want to try that, run it by support, see what they are thinking,
[21:19] maybe we had had a lot of people asking. I know we had been doing text testing. It's just
[21:25] sometimes it's just a question of, are we ready to call it risk free or risk enough free and how much risk do you want to carry?
[21:34] And are you ready to reinstall everything if go things go horribly wrong in six months? It's it's manageable.
[21:40] Yeah. I had some discussion with with the admins and
[21:45] and
[21:46] yeah. I mean, Red Hat Linux nine is different to
[21:50] on on on the surface. It's a little bit different than eight ten.
[21:56] They told me. So yeah. And that's why and be again,
[22:00] if if there was an in place upgrade
[22:04] between eight, ten, and nine,
[22:06] then chances are that SQL also serve survive that update properly. Mhmm. I just don't don't know that. I'm not saying that they need to do the update, but also,
[22:16] in my experience, update and fresh install can be a little different in the libraries.
[22:22] Yeah. Yeah. So I would say go for the Doctor model and use that. It's not designed for that, but I find it the best and easiest way to actually do that.
[22:33] All it requires is one more database.
[22:36] Yeah. Yeah. Okay. That's not the problem, I think. And as and it also gives you something else.
[22:43] This will allow you to actually
[22:46] make sure that all the IPs are reachable, that all of the load balancers are properly set already and so on. So you don't need to scramble in the last minute when you are replacing all these IPs and whatnot. That you don't have, for example, a storage area
[22:59] which has an IP restriction that no one remembers about until you go live.
[23:05] Been there. Done that.
[23:07] Yeah. Yeah. Yeah. I was on the phone at 3AM in the morning. Someone was screaming and, well, nothing is working. Turned out their storage area was whitelisting the access,
[23:16] and they forgot that they changed the IP. So I I I don't know. I mean, plus, this way, if something is horribly wrong,
[23:25] you can still swing back. Not that you cannot with the old one,
[23:29] but, you know, just Yeah.
[23:33] I I'm the person that makes three plans for back for for
[23:37] if something goes horribly wrong. And if I don't use any of them, I'm lucky and I'm happy, but I will still have three plans every time I do something.
[23:47] Which other good planning than
[23:49] right?
[23:50] Yeah. Yeah. We have we have on production. We have, 1,100
[23:54] accounts.
[23:55] So it's,
[23:57] I have to be careful what I do.
[24:00] That's clear for me. So I'm also the the the the kind of type of of guy who's making,
[24:07] maybe three plans, not only one. Yeah.
[24:10] Yep. Okay. Thank you very much. Awesome. So, Ed, I see you. I have two questions in the chat before you, and then it's you.
[24:18] Okay. Edie,
[24:20] archiving.
[24:23] So archiving for resubmitting.
[24:26] So for the setting inside of the route step, the defaults cannot be changed.
[24:31] So I know they are annoying. So for the ones that don't know what I'm talking about, and now I'll just swing to my server because I really, really want to go to my server.
[24:40] So give me a second to find where my server is hiding now.
[24:48] Let me know when you see it or if someone doesn't mind telling me.
[24:52] Yep. So awesome.
[24:55] Hey. See? Security kicked me out.
[24:59] I really need to do something with it.
[25:03] Yeah. So
[25:05] so
[25:06] on a route step,
[25:08] as you may or may not know.
[25:13] Hold on a second.
[25:20] On the send to partner step,
[25:23] at the very bottom, you have the two checkboxes
[25:27] on our high files on success and on the failure where the default is basically default.
[25:32] That's what it is.
[25:34] And this will be determined
[25:36] based on the global configurations.
[25:40] You cannot change the default here to be either enabled or disabled by default.
[25:45] And,
[25:46] Edi, by the way, that's a request I'm getting quite often from different people. So if you don't mind opening an idea for us, Ted, because that might be an easy change to do. But it is the same, like, the retry settings over here, which we all know are really bad because this is three seconds and two seconds. Right?
[26:06] And and that's why I wanted to open that just to show because in addition to changing that,
[26:12] don't forget when you're doing that that these retry settings usually also need fixing
[26:18] because this is five times, three seconds, two seconds.
[26:22] So everything goes within a minute. Who wants five retries in one minute?
[26:26] No one. Right? So don't forget to update those as well.
[26:31] But back to your question, there is no way to to change the defaults on these pages. There is no configuration for that to open a open an idea for Astrid to see what they can do with the server configuration.
[26:45] But in the meantime,
[26:47] you can always use the API to just list all of the steps and make sure that all of them look exactly as they are supposed to.
[26:55] Right?
[26:56] So you cannot do some anything on the front end, but if you are running a report nightly, for example,
[27:02] you can
[27:04] find them quickly.
[27:05] That's what I would do if I were you and if you want to enforce it
[27:09] just because we really cannot. And that's one of the things I also had pointed to Remy when they are rebuilding all of that to see what we can do about that. Talking about that,
[27:21] and it's this is not much change in the new UI, by the way, even though it will swing in. Right? So sorry about that.
[27:29] This is where actually using templates might help you. Because if you can stick this step in a template with some parameters,
[27:38] then no one needs to recreate it. Right?
[27:41] Mhmm.
[27:43] Instead of building this step by step on every route, if you actually use a template route and just feed it parameters
[27:50] with the site name or whatever else you need,
[27:54] then you don't need to worry about who sets what because it will be set in a single place.
[28:01] K.
[28:02] Right? Yes. Yes. So okay. And I saw so hold on a second. I lost my chat window. Let me give me a second. I have a technical difficulty looking at both of them at the same time. So let me look another device.
[28:18] And you're asking where the files are being stored. Right?
[28:24] Yeah. In the configuration
[28:26] to the for the global configuration.
[28:29] Mhmm.
[28:30] Just based on what you said, it sounds like I will set
[28:34] it to
[28:36] disable.
[28:38] And then
[28:39] when we create the route step, we will
[28:43] enable
[28:44] the
[28:45] on on failure.
[28:47] Yes.
[28:48] Yeah.
[28:50] Yeah. So we're yeah.
[28:51] Like, here, the disabled is the global the global policy.
[28:57] Yes. And then yeah.
[28:59] Okay.
[29:00] And this policy is essentially for the inbound files. So the ones on down on the
[29:07] steps are
[29:09] a little different.
[29:11] That's why template will be your best friend.
[29:15] K.
[29:16] And then the archive folder
[29:18] really shouldn't should not be the home folder of the users in any way or form. It should be on a different drive altogether
[29:24] because it's on our drive altogether. Okay. So it you know, if you put it on the same drive and you have a drive failure, you're losing not just the subscription folder, but you're also losing the archive. So as a rule,
[29:35] the archive folder
[29:37] should be on a separate drive altogether. So if your data is in slash data
[29:43] or slash data life or whatever, slash data archive, if it is on a different media, it's also better. So I would if you and if you have one very good one and one crappy NFS somewhere that barely works, the barely works one goes for the archiving.
[30:01] But the point is So
[30:05] Annie, our our use case is going to be
[30:08] it's it's gonna be temporary.
[30:10] So we
[30:11] we are not going to you know, we
[30:14] advertise that
[30:17] ST is a pass through. We it's not a file storage.
[30:22] Really and
[30:23] we're only going to hold them for five days. Because if there's an outage,
[30:30] they the the issue will be resolved,
[30:34] you know, within well,
[30:36] historically,
[30:37] it's been resolved within a day, and then we just
[30:41] trans resubmit the transfers that failed,
[30:44] and then we move on. So
[30:47] Here here is where you control that. You have delete files over then where you specify how long to keep, and there is an archive
[30:54] maintenance
[30:55] application in the applications list that basically runs the schedule. So every minute, it will go and delete everything. Those will be the files, so you don't need to worry about them. We'll self clean after ourselves.
[31:08] That way, you can set up the subscription
[31:10] file itself on success or on failure to delete the file, so it disappears from the subscription folder.
[31:17] The rule of engagement is that
[31:19] if there is an archive copy, we'll run with it anyway.
[31:23] So as long as you are archiving the files you might want to resubmit,
[31:27] you can always delete the subscription folder copy because we will never use it anyway
[31:33] if you have an archive.
[31:36] Yes.
[31:37] And this archive is so you can also don't forget you can encrypt it if you if you want to make it more secure. It doesn't
[31:47] change.
[31:48] It doesn't give you it it there is a performance hit, obviously, but we're talking about milliseconds here
[31:54] because it's a one time. It's not like repository encryption where we need to encrypt the crypt and link, you know, like crazy.
[32:01] It's and that's why I'm saying this storage doesn't need to be as good. It still need to be shared storage and so on. But, technically speaking, we write once when we write the file, and we read it only if we need the file for a resubmit or to delete it.
[32:17] So there is a lot less volume going on on this drive.
[32:21] Okay. So
[32:22] it it's and this being on a separate drive gives you both security.
[32:29] And,
[32:30] you know, because if something goes horribly wrong, you'll lose your storage. You have all the files somewhere, so you can resubmit from there still even if you are rebuilding the home folders.
[32:39] Yeah. Hey, Annie. You know what? That exact
[32:42] topic on the separate drive, I was told that when we first stood up on SecureTransport
[32:48] and then about a month ago, I'm finding out that we had to put it on a separate drive because
[32:54] when the archive does archive at the same time that the file copies to its destination,
[33:00] that that copying
[33:03] at the same time on the same drive, it will air out.
[33:07] And when you have it on separate drives, it resolves that issue. That
[33:12] that's something that I went through just a month ago. Yeah. And the more volume you have, the more likely it will happen because we are writing two copies at the same time, and a lot of drives are just like, no. What the hell you are doing? You need to copy later or something. Exactly.
[33:26] And it's it's down to your solution. But, also, if you are carrying a huge volume of data,
[33:32] doubling the load during writing,
[33:35] which is the most precarious part of the process,
[33:38] right, is really a bad idea.
[33:40] Because once you write the file, once we have the file, we can retry. We can resubmit. We can do it anyway,
[33:48] each way, even if something goes wrong. But if we haven't got the file yet
[33:53] sorry.
[33:54] You know? Yeah. No. And and do
[33:58] not encrypt the the
[34:00] that do not encrypt
[34:02] the file that is archived will not be encrypted. So there might be a data at rest policy
[34:08] where that you might wanna encrypt that. Although, you're right. If if it's there for me a day, maybe some policies allow data to be unencrypted on secure transport,
[34:18] but but it is unencrypted. It's not encrypted at that point.
[34:22] So if you say do not encrypt,
[34:24] the file is saved as is.
[34:27] So safe as is meaning what? Meaning safe as is what? It's not encrypted.
[34:32] No. Correct. So if if it is PGP encrypted, it will be PGP encrypted, but we are not going to put another encryption on top of it. So The PGP. If Yeah. You if the file so if it is the inbound archiving,
[34:44] the file arrives from the partner, we save it exactly as it is coming from the partner.
[34:49] If it is the outbound one, we save the file exactly as we're sending to the partner.
[34:54] Exactly. If you enable
[34:56] the encryption encrypted. Yeah. Talk, please. Yeah.
[35:02] If it is if if you want to secure it on the file system,
[35:08] then you need to encrypt it by selecting the certificate over here. Got it. Yeah. Exactly. Exactly. Yep. K. Edi, do you have any more on that?
[35:17] Yeah. So just to clarify. So the files
[35:20] delete files
[35:22] older than.
[35:23] So is it gonna delete all files
[35:26] in every subscription
[35:28] or just the files that were that failed to reach its destination?
[35:33] So this have nothing to do with the subscription folders themselves. This will be deleting in the archive folders only.
[35:40] So you have two separate processes you need to enable for clinical testing. So there are two separate applications.
[35:46] One
[35:47] of them is the archiving application,
[35:50] which is taking off from this parameter,
[35:52] and I don't have it created, I think. So oh, here it is.
[35:57] And this one, if you open it, you'll see that it basically, all it has is a scheduler.
[36:03] It doesn't have anything else because it gets its parameters from the archiving parameters,
[36:08] and there is some of them can be over on a business unit level or account level and so on. But the point is that that parameter for the one day only is for the archiving folder itself.
[36:20] If you want to clean the subscription folders as well,
[36:24] you need to have the file maintenance application.
[36:29] Yes.
[36:31] Which where is my file maintenance application now?
[36:36] Here
[36:38] it is. Yeah.
[36:43] Is this the one I wanted?
[36:45] Yeah. Where you specify what files to delete, and those will be for the files from the subscription folder, the one that your users can actually see.
[36:55] So
[36:56] if you do not have delete on success, delete on error on the subscription itself,
[37:01] then and you have this enabled for seven days, and in seven days, will delete everything from the folder. You said you don't keep more than five,
[37:07] So what I would say is that if you want, you can keep everything for one day and then set up just want to clean everything two days or older. You know?
[37:16] Or you can just do delete on success error and rely on the archiving for the resubmit if they don't need to delete it. Any what's the application name
[37:26] on
[37:27] top of that? Oh, file maintenance. Okay. Thank you. Thank you.
[37:32] So
[37:33] so, Iggy, it's up to you how you want to design, but don't mix up the two of them this
[37:39] one is dealing with the subscription
[37:41] folders and account folders.
[37:43] This is what the data users see.
[37:46] The archiving and the days over are for the archiving folder only.
[37:50] So because when you enable archiving, there are two separate copies now. Right?
[37:54] One in the subscription folder, one in archiving. That is both for advanced routing and also for basic apps.
[38:00] But on every subscription,
[38:02] at the very bottom of every subscription,
[38:06] plus I don't have any subscriptions because
[38:09] it's a new server and I haven't played with that. So it doesn't matter what type it is. You know at the bottom where you have post routing settings?
[38:18] Or they are called just post settings on the basic app, you can delete on failure or success. This will delete automatically once you come out of sending.
[38:28] Mhmm. And if you have your
[38:30] so for basic app, if you delete on error, you will not be able to resubmit
[38:35] the outbound because there is no way to archive
[38:39] on the outbound on basic app.
[38:41] So you only need to resubmit from the inbound and with because of volumes that might not work very well.
[38:48] So in basic app, you might want to delete on success,
[38:52] but leave it like that on failure and rely on the file maintenance to actually clean it up.
[38:57] Right? Mhmm.
[38:59] For advanced routing,
[39:00] you can safely delete both on success and failure
[39:05] because you have a backup copy with one warning. If you have multiple steps and you never reach the sent to partner and you delete on error on failure,
[39:17] you don't have a backup on the outbound. So you only need to start something inbound.
[39:21] So you might want to look at what exactly you're doing. So if it is just sending,
[39:26] you can delete safely both of them. But if you have other steps
[39:31] inside
[39:32] of
[39:33] of the route,
[39:35] for example, PGP encryption, and if the PGP encryption fails, the routing fails, so this will kick in. But you don't have a copy because you never got to send to partner.
[39:46] Makes sense? Yes.
[39:49] So design it based on what makes the most sense for you. And then when you enable the archiving and the file maintenance applications to actually run,
[39:58] especially in your environment, you need to run nightly, but clean up files that are two, three days old.
[40:04] This will keep your server really, really clear.
[40:07] And the good news is that when the file maintenance one starts, it doesn't care why the file is in the subscription folder. All it cares about is the date.
[40:16] Okay.
[40:17] So it will just wipe everything it sees.
[40:21] Okay. So importantly and more importantly, sorry, it will also clean all of the STFS folders for it, all of the additional pieces we have put for this file. So it it's intelligent
[40:33] enough to not clean up for the file.
[40:36] And
[40:37] what it will not do is to clean orphan STFS files. So if someone is going on the OS level with deleting files because they like it
[40:45] and not deleting STFSs,
[40:47] the file maintenance application will not clean this up because it finds a file. Oh, Annie dot TXT is about to be deleted. So I'll go and find all of the other files that belong to this file, and I'll delete them.
[41:00] Makes sense? Yes.
[41:02] Good. Okay. So, Annie, if can we go back to the application
[41:10] archiving?
[41:11] The
[41:11] the archiving
[41:13] the application or the setup page?
[41:17] The for the global settings where where we're gonna set the global settings? Yes. That's the it's the setup. Okay? Yeah. That's what I'm asking.
[41:26] Yeah. Sorry. I No. I'm doing the archive folder.
[41:31] So when a file
[41:33] when when I set this,
[41:35] it's gonna be a Windows
[41:37] it's gonna be a Windows
[41:39] server.
[41:40] Okay. So you can do so you can either do slash slash server name
[41:46] archive.
[41:47] Right? Alright.
[41:49] Or you can do
[41:53] Mhmm. You know?
[41:55] It's a Windows, so you can do either. Same like whole folders, same like everything.
[42:01] So
[42:02] when it gets this folder and you know? So what happens is that when the file counts for account Annie with the login Annie, it will create a subfolder called Annie for the account name, then a subfolder called Annie for the login name, and then subfolders for each folder
[42:17] that is under under the home folder of the user where you are hiding from. So if you look at the structure, you'll actually see all of the files in separate subdirectories,
[42:27] and you'll know who they belong to. But the file gets a new name
[42:31] as well. So that if you need to archive the same file name twice, they don't step on each other toes.
[42:38] Okay.
[42:40] Okay. And then so when a file fails
[42:43] to reach its destination,
[42:45] that folder is automatically
[42:47] created in this path and then the file name
[42:51] is changed?
[42:53] The file name of the copy is changed to a separate value. And if you look at the tracking table where the resubmit works from, we actually will inside of it is there is a note at the bottom that gives you the full path to the archived file so we know where we put the file.
[43:08] Okay.
[43:09] So and that's why tracking table entry is a requirement for resubmit because that's the only thing that knows what which file you are trying to resubmit.
[43:18] So in the tracking table, when you have this enabled, you'll see at the very bottom,
[43:23] now there will be the additional
[43:26] line that says archive
[43:28] file and then a path next to it,
[43:31] and that's what we'll be using to try to get the file. If it's not there, then we'll go to the subscription folder itself,
[43:38] you know, what you had already seen with basic apps.
[43:40] Okay. And then just to clarify, the whole reason that we're doing this is because we have a lot of routes that have multiple steps,
[43:48] and then
[43:49] it fails to reach one
[43:52] reach one of the destinations within the step.
[43:55] When we resubmit the incoming, it goes through the whole steps again.
[44:00] But we just want the one the one or two destinations
[44:05] that failed
[44:06] to deliver.
[44:07] And so when we hit that that submit
[44:10] on that failure, that's the only file or the only step that will be triggered. Is that correct?
[44:17] Yes. This is why we have those outbound
[44:21] copies
[44:23] on the send to partner at the bottom because they allow you to actually make a copy of the file as is the already transformed file.
[44:33] So it can be just resubmitted on its own without you resending to the other five people and annoying the hell out of them in the process. Right.
[44:41] Or
[44:43] causing causing
[44:44] failures
[44:45] on a duplicate file being delivered. And we have we have that as well in our environment.
[44:50] So okay. I think I've got enough to
[44:53] to complete my task. Thank you very much. Okay. So there are a few questions in the chat, Ed. I haven't forgotten you, but there are a couple of follow-up questions on this one that I'm seeing.
[45:04] So
[45:06] so what the rotate folder is about, we actually have a very nice
[45:14] so
[45:15] if you have a lot of files that so what it this will do
[45:19] is
[45:24] So by default, it creates one folder
[45:28] for each folder inside of the user's folder.
[45:31] However,
[45:33] if
[45:34] there is a lot of files,
[45:36] that means you might have thousands of files or multiple thousands of files in the same folder,
[45:43] which can be a problem for reading this folder or even opening it. So rotate will actually create subfolders per day or even per hour based on volume
[45:51] so that you have longer paths in the archive folder, but smaller folders to deal with.
[45:58] It's just one of those precautions. If you're on a low level environment,
[46:02] in a low volume environment, it doesn't matter.
[46:06] It's safe to always enable it though because ST is kinda intelligent on when to use it. But that that's what this is all about.
[46:14] It's time stamp folders as opposed for daily and hourly as opposed to just one folder for all files from the folder.
[46:20] Thanks, Amy. Thanks. Yep. Okay.
[46:24] So back to the questions in the chat. Edi, anything else on this one or we said?
[46:30] Yeah. I think I've got it. Thank you very much. Okay.
[46:34] And I had
[46:36] the default archive global settings or business unit row settings. The default
[46:42] so, Chris,
[46:44] default
[46:45] so the rule is account beats business units beats global.
[46:49] So if they defer,
[46:51] if the account has specific settings, they'll always work.
[46:54] If not, it checks if there is business unit. If not, it goes down to the global. So it's a
[46:59] slight draw, and it's valid for pretty much anything on the server.
[47:09] Okay.
[47:11] So hold on a second. I saw another question.
[47:17] Let me stop sharing so I can actually see the name of the person asking the question. Sorry about that, guys. Single so
[47:25] Vinay.
[47:26] Sorry. Oh, yeah. This is Vinay here. So I heard like I I I really mess up names, So I I need to apologize.
[47:34] Me and names are not good friends. So I'm not sure what you are asking. I apologize. I don't understand the question.
[47:41] I've seen, like, enterprise
[47:43] architect group usually work on domain plus to model. Right? They have certain finance or
[47:49] banking
[47:51] or some Mhmm. Marketing kind of different domains, and then certain APIs falls under those domains. Right?
[47:58] So is there any group level arrangement we can consider here
[48:03] while designing the
[48:05] APIs and deploying it into the right
[48:08] category
[48:09] so that we can see them in the
[48:13] high level picture, like, these are all the domain clusters and all API residing to those API together?
[48:20] So not from the ST itself.
[48:23] ST is having its own API. However, if you put an API manager and gateway in front of it, which is strongly recommended, it doesn't need to be an hours,
[48:32] but we strongly recommend to put one. There, you can do whatever makes the most sense for you, and you can do whatever you need to. From an ST perspective,
[48:43] these days, it really doesn't even may doesn't even matter what kind of cluster it is. The cluster is application level technically.
[48:52] So for all intents and purposes, it's going to the same place.
[48:55] Mhmm.
[48:56] So it doesn't really apply in the same way. However, if you want to do your own enterprise domain cluster models to do something specific
[49:05] upon the API level for performance or for security or what you want to allow people to do.
[49:12] It's up to you.
[49:14] But from the ST perspective, we really don't care.
[49:17] I cannot really think.
[49:19] Okay. Mhmm. So I have quite a lot of questions left in the chat, but Ed had his hand up about this time. So Ed?
[49:28] Yep. Hey, Annie. How are you?
[49:30] Okay.
[49:31] Hey. It's a new year again.
[49:34] I know. Hey. So I had a I had some questions around the new database as it relates to standard cluster.
[49:44] We're we're on the we're on the August patch right now and and have been playing around with the September patch, which introduces the new database. But I guess my first the the one one question I have really is is are are we now able to add more than two nodes with a standard cluster with this new database?
[50:04] Three officially.
[50:06] So the official line still stands. You can have up to three,
[50:10] but the third one
[50:12] might have performance considerations.
[50:15] And because the primary will carry the database for all of them,
[50:20] it's a lot more likely to actually have a problem.
[50:24] Okay. Okay.
[50:25] Because it yeah. It seemed it seemed like this would be scalable now, but I guess I guess the the the same holds true with standard cluster anyway. So so it technically scalable.
[50:37] The problem, the bottleneck is that literally there is a database living on the primary now.
[50:43] So your bottleneck,
[50:45] it might even be worse than before
[50:48] because we removed all the communication that was causing the you cannot add more notes. But now every single note actually works off the primary database.
[50:57] So you have pretty much all this big services and the database on the same note.
[51:02] That's your bottleneck now.
[51:04] I see.
[51:05] Which also means, and that's basically that's important for
[51:09] people with standard clusters,
[51:11] is that if you are already
[51:13] on on on the verge of overloading your standard clusters,
[51:18] you will have a problem when you switch to the new standard cluster because
[51:23] the load on this primary is increasing because of the new database. Because now all the secondaries, all two of them, if you have two or one,
[51:32] instead of working off its own database,
[51:35] are working off the database of the primary
[51:38] in real time.
[51:40] Mhmm.
[51:40] So you don't have the dispatcher anymore,
[51:43] but now you have two nodes talking to the same database.
[51:46] That's your bottleneck.
[51:49] I see. I see.
[51:51] Really trying to push everyone to enterprise cluster.
[51:56] It's it's not that. It's really about resource management. Forget about
[52:01] anything else. The point is that the moment when you have your application in your database in the same server, you are running the risk of things going horribly wrong at any time when you have overloaded it.
[52:13] Right?
[52:14] Enterprise
[52:14] cluster
[52:16] allows you to split the database out, and you can scale it as much as you can.
[52:20] It's
[52:22] technologically,
[52:23] both clusters are behaving the same way now.
[52:27] It's now down to resources and risk management.
[52:32] So Okay.
[52:33] And and just real quickly then. So
[52:37] if I have a two node cluster
[52:40] and
[52:42] I need to take one of the if if I need to take the primary down, right, I've noticed that Yeah.
[52:49] You can't you can't do that unless unless,
[52:52] I guess, you forcefully
[52:53] do it,
[52:54] to switch over to the other database. But is that is that how you would do it? So say I have say I have two different say I have two servers in my in my standard cluster. Right? I wanna bring one down for maintenance purposes or what have you, and that happens to be the primary.
[53:08] Can I do that proactively?
[53:10] Do I have to stop
[53:12] do I have to force stop the database so so the secondary takes over as the primary? Or how how does that how does that work?
[53:20] Technically speaking,
[53:22] the data bay the the moment when the TM disappears,
[53:25] it's supposed to be switching to the other database anyway.
[53:28] So it's still you need a live TM for the database to be the active one.
[53:33] Uh-huh. But there is some control that may or may not work. I honestly, anytime I want to do something like that, it'll do a full stop all,
[53:41] which will force it into the other database.
[53:45] The other days is When they would stop all on the primary,
[53:49] it it tells you the database
[53:52] can't be stopped. Is that is that a is that an accurate statement or no?
[53:57] It shouldn't be happening.
[54:00] Okay.
[54:01] Technically speaking, you should be able to. So if you are still having problems or if it's really misbehaving,
[54:08] I would say open a ticket, talk to support.
[54:11] We might need some documentation.
[54:12] We might need to play with it a little bit more.
[54:15] It cannot be stopped because there is the external connections, but it's supposed to be stoppable.
[54:21] So Okay.
[54:23] I I I don't know if we didn't put some protections we forgot about or I didn't know about.
[54:28] I know that when I was playing with that, I was able to shut down everything, but then I I played on a lower environment where who knows what else was not the same. Right?
[54:38] Right. But other from that
[54:40] so the good news is that the other database
[54:43] is fully updated except for the server logs that, you know, don't get synced anyway.
[54:49] So the moment when you switch to the other database, it continues from where it stopped. So it's pretty straightforward,
[54:56] a lot cleaner than the mod the old model was.
[54:59] Right.
[55:00] Yep. Yep.
[55:02] Okay.
[55:04] I think We can do.
[55:07] Yeah. I think that's that's it. I know I know Merrick's on the call. He's he works with me at Dell. Merrick, did I miss something? I don't think we had
[55:16] is there anything else as it relates to that?
[55:19] Yeah.
[55:20] Yeah. The the last last piece of the the Postgres update that we noticed is
[55:25] some of the the configuration files that we used to have that we play around with,
[55:29] namely the internal db.com,
[55:31] previously the
[55:32] my ssqdb.com
[55:33] file.
[55:34] I it seems that that file doesn't exist anymore. Was it renamed or moved or
[55:40] basically,
[55:40] any any tuning that we used to do for the the MariaDB and MySQL database,
[55:45] how do we, I guess, apply that to Postgres, or is there any updated
[55:49] tuning?
[55:51] Yeah.
[55:52] You mean things like number of connections and so on. Right. Table and cache stuff like that. Yeah. There there is a configuration file somewhere in there. I don't know the name of it off the top of my head.
[56:04] I'll look after the meeting, see if I can spot it. If not, just open ask support. They'll give you the name.
[56:11] What I would say is open a ticket and just ask them how you control the number of connections now because that's what will tell you where this is. We have that multi to the UI. I don't think we will at this point or at least not all of it,
[56:25] but it's different because post grade self is different.
[56:29] When we did the MySQL to Maria, almost nothing changed be besides the rename of the file because Maria was a copy of MySQL
[56:36] as we all know. Post grade is a totally different database.
[56:41] But the old tuning will not work.
[56:43] So you still can update some things, but it's different.
[56:48] So if Right.
[56:50] And we don't have the white papers and the documentation yet on the tuning.
[56:57] So what whatever you come up with, I would say to clarify with support to make sure that they don't know about something that should work differently.
[57:05] The good news is that we actually know a lot about Postgre
[57:08] because it is one of our easy databases.
[57:11] So
[57:13] it's the same tuning. So if you're looking for a tuning,
[57:17] look at the post grade tuning for enterprise cluster because it will be pretty much the same here.
[57:23] Okay. And and then I guess I guess finally,
[57:25] regarding the enterprise cluster, we we have an environment that is Oracle database. Mhmm.
[57:31] Applying the
[57:32] the
[57:33] the September, October, you know, December patch,
[57:36] that won't necessarily have any significant changes to an enterprise cluster using Oracle.
[57:41] This will only be for anything using the Not for the Oracle, but for the edges on this environment
[57:47] will go to Postgres, so you'll need to deal with the new ports and the new communication.
[57:53] If your edges are synchronizing,
[57:57] you will be switching to the new model of synchronization.
[58:00] If they are standalones, nothing will change besides the fact that it's a new database with new ports and so on, so it's easier.
[58:07] But if you're synchronizing your edges,
[58:09] the edges will also work like the standard cluster with all of them talking from the state taking from the primary.
[58:16] Got it.
[58:18] But that's pretty much it.
[58:21] Okay. Yeah. I think that was that was all my other questions, Ed, for Postgres.
[58:27] Okay.
[58:28] Okay. Back to the chat, which is very busy today, so I will go in order. So Mirza,
[58:35] you want to update the edges to the December build.
[58:39] What build are you on at the moment?
[58:43] Yeah. I was on
[58:46] November build before I tried to update to
[58:50] December '1 in
[58:52] Yeah.
[58:53] It requires
[58:55] new cost.
[58:56] It did not synchronize. I could not update my
[59:00] edge server. I was able to do my primary server on
[59:04] on my I mean, front
[59:06] server back end
[59:08] servers, but I could not update my edge server, which is which are in the DMD.
[59:13] I was able to do one, but when I tried to do second one, slave,
[59:18] it it had a hard time to communicate with the with the primary edge server. Did you did you open the new ports that are needed between them?
[59:28] No. I still working with my firewall team.
[59:32] That's
[59:34] I learned during update that
[59:37] Okay. Both need to be updated. So so But they should they should have given warning.
[59:43] Okay. Before you proceed to this update, make sure you have opened these ports in the firewall.
[59:49] I think that they added it to in September somewhere when we push it for the first time.
[59:55] I will pass back the message that it should actually be in everyone.
[59:59] Yeah. It should be highlighted with the big big red ink.
[60:03] It I think that they missed some of the updates. I'll verify again. I'll talk to the document, see what we can do. But yes. So if some if you're updating from anything before September,
[60:16] then you you have to switch to the databases, and there is a new requirement for if you are good to install, there is new user needed,
[60:24] but also new ports.
[60:26] We had updated the admin guide. It has a section about ports needed, and it had been fully updated now.
[60:33] So if you look at that, you'll see the communication ports needed over there.
[60:38] But I'll pass the message that the release note also should be highlighting
[60:42] if Yeah. It should from the folder. Yeah. It should be saying before you proceed, please make sure these ports are open. Something like Okay.
[60:52] It's easy to miss. Right? Yep. It it is. And a lot of people did miss it when when they changed, and I've been Yeah. Answering the same question over and over, and I'll pass the message again.
[61:04] Yeah. Because my team is not easy to go through the firewall team. It's it's it's
[61:11] it's internal pain.
[61:13] I I fully understand.
[61:16] Yeah. So I'll get back. So okay?
[61:19] Good. Anders, my second question with that is about
[61:24] how can you show how to split the
[61:27] DB files into
[61:29] text format? So
[61:31] one of our team
[61:33] security team want to
[61:36] want to have a instead of in the DB file, they also want to have files in the text format so they can read them
[61:43] ahead of time.
[61:45] Who's logging in was Oh, for the login. Oh. Yeah. For the login. Yeah. So what if you're Can you show that
[61:52] Yeah. Okay. From your server?
[61:55] Yeah. I will in a second.
[61:57] Okay. Thank you. So while I'm sharing again,
[62:01] I have one more question about the standard cluster that so,
[62:05] Jacob.
[62:08] Schedule and folder monitor
[62:10] start on all the notes, but they are active only on the primary
[62:15] unless you shut them down there or something like that. So not your changes
[62:20] compared to the old way the servers were working. The services will start and get suspended on the secondaries.
[62:27] It should be running only on the primary.
[62:31] And we're talking about folder monitor schedule or, you know, the two actions. But also, don't forget that now they are literally on the first page of the server. Let me get back to my server.
[62:44] Don't forget that they are over here as well. Okay. I don't want to save anything.
[62:49] But so you can control who is running what by schedule folder monitor. So you can stop them on the primary start on the secondary if you really want to or something like that because they're independent on the servers.
[63:02] But this is who is running the service, and if they stop on the primary, the secondary need to pick them up and to should pick them up and usually will pick them up as long as the service is running. When it says running here,
[63:14] it does not mean this is who is running the service. That means the service is up here. It might still be suspended.
[63:20] You'll need to go to the locks for that.
[63:23] Gotcha. Okay.
[63:24] Okay? So in a properly working server cluster,
[63:29] you should be seeing running on all nodes here.
[63:33] But when you go to the server lock during the startup time,
[63:37] on the second reason where they are not running, you will see the line saying folder monitor suspended
[63:42] because there is a parent running something like that. I don't forget I I don't remember the exact message.
[63:47] Makes sense? I'll take a look for that. Yeah. Thank you. And
[63:50] it only shows up on startup
[63:53] when you start the TM
[63:56] because that's the time when you make the termination. And if you lose it on the other server, there will be another message that says
[64:02] exiting suspended state,
[64:04] something like that when it picks up. But it will be running on one node only.
[64:09] Again, service runs on all nodes.
[64:12] The pros process runs on all nodes. The actual action happens on one node.
[64:19] Okay. That makes sense. Sense? Thank you. Yep. Yep. Mhmm. Okay.
[64:23] Mirza, question. So you were asking how to change the server lock to files. So it's actually not on the admin UI.
[64:31] So
[64:33] this is the documentation of the admin guide, and I'm actually in a very special page. I'll leave it here because I want to come back to that. So let me open another
[64:43] very quickly for that.
[64:47] So server
[64:49] were
[64:50] locked.
[65:03] So Log four j files.
[65:08] So see this section over here, Redirect Log four j output from the database?
[65:13] Yeah. I can see it. Yeah.
[65:15] So this here explains how to do so. This is the appender. It needs to be replaced to another one. So you need to add another appender,
[65:23] which is the rolling appender, which will put it on the file system.
[65:26] And then when you go to the lower
[65:30] so you can either replace it or you actually can add another one. And lower,
[65:36] there is another section
[65:38] called logger. Inside of the logger, you specify which append is being used, and you can use more than one.
[65:46] So you can either switch
[65:48] only to to this to to here, or you just add another one called server file and then just add to any logger you want to go into the file as well.
[65:57] If it's all the I I want to have a DB is
[66:01] the DB, but, also,
[66:03] I want the DB also show the the text file too for for my Okay. In which case
[66:10] so in each each case, instead of replay changing,
[66:13] you want to add another append under the other one. So you have one called server lock. You add another one with a different name
[66:21] called server lock files or something, and then you change every logger that requires the appenders. So if you look at the file at the log four j file, it has the appenders at the top, loggers at the bottom. And don't forget that you have a log four j file for every service.
[66:37] So one for s h d, one for FTPD, one for t m, one for admin.
[66:41] You need to change all of them or all the one that you want to re to resend.
[66:46] Yeah. Like a
[66:48] the the only I guess,
[66:51] is
[66:52] in the DB is only the
[66:55] let's say, if someone is admin and try to log as admin
[66:59] Mhmm. Then all the failing information
[67:03] or connecting information goes into
[67:05] into DB. Right? It doesn't go into
[67:08] text.
[67:09] Yes. So that that so
[67:11] that part which doesn't go to
[67:13] text, that that's the only part I want to Yes. Couldn't So okay. So look at the file. Open the file, the admin Log four j or the TM Log four j, whenever you whichever. See if you can spot the logger you need. They're very descriptive.
[67:27] If you cannot explain
[67:29] to support exactly what you're trying to do, and they'll tell you what to change.
[67:34] Okay.
[67:35] So that because
[67:37] Mhmm. They're very granular.
[67:38] You might not be able to move only those logs or copy those logs,
[67:43] but at least we can help you minimize
[67:46] the number of things you are sending.
[67:48] Yeah. Because I
[67:51] think it make a mess to write too many text files. Yeah. So I don't want to have too many text file, only
[67:57] what the security team is looking for. Yep. Yep. Talk again, look at the Log four j files. It's standard Log four j. This x section shows you how to do a server log. So you'll need to name it something different because it the appenders need to be,
[68:13] unique. So if you're going to keep the database one, you just want to add this one with this initial name and just add it whenever it's needed.
[68:21] Sure. And it it can be done. Right? Yes.
[68:24] It's
[68:25] I'm doing it on every server I ever run. Oh, okay. Good. So it's easy enough to be done. I would still keep it in the database. That's why I usually do the double. It doesn't add to the perform it doesn't have a performance hit too much. It might be a little
[68:40] you know, there is milliseconds here and there, but not enough to actually be a problem because Log four j is very good at double streaming.
[68:48] Yeah. Yeah. Because this way, it helps. I think on the old version, we could not do it. Right? Oh, you could. Oh, you we could? Okay. You also I
[68:57] called in, and they they could not help me with that. Right? That time Yeah.
[69:03] It may I Yeah.
[69:05] So that's one thing.
[69:07] When you ask the question to support, make sure that they actually understand the exact question. Because if you want and ask them how to move only a specific
[69:15] part of the locks, the answer will be you cannot because they are not split that way.
[69:20] Right? Yeah. So it still might be that we cannot move only what you need. You might need to move a lot more than that or copy a lot more than that, and that is yes. But if you go and ask them, can I just put the connection information in the log files and nothing else, the answer is you cannot do it? Because the logger that is doing that does other things as well. And we can just move a whole logger in and out.
[69:45] Makes sense?
[69:46] Okay. So just I have I have to ask them a specific
[69:50] Yeah. Just make sure that when they tell you it cannot be done, that you actually explore further what exactly cannot be done.
[69:59] Because it's a correct answer, and when you come with a specific question, cannot be done is a valid answer. Right? Yeah. But because
[70:08] the expectation is that what you are asking is what they what, you know, what you need. Okay.
[70:16] Okay. That's a very long name. Maria de Los Angeles.
[70:20] I'm sorry. I don't even know how long of those names I need to read, and I'm really bad at that. It is a composer's name. It's Maria de Los Angeles.
[70:29] Oh, okay. So I stopped where I had me. Yay. Yes. It's just another Spanish name that is very traditional.
[70:36] I I I I can recognize it, but also I know that in a lot of those names, you actually have some of the other names part of the composite name. So I I I I'm trying not to offend people, so I apologize.
[70:50] I mess up names. Okay.
[70:52] So Call me Maria. Call me Maria.
[70:55] That's okay. I can go with the whole one. Don't worry about it. So regarding the time, you have a problem when you have a huge amount of files arriving at the same time
[71:04] or when you have a lot of files that error ring.
[71:07] So are those explain.
[71:09] Let me explain. The partners, okay, sometimes are once, another time, chat others.
[71:15] All which are indirected.
[71:17] Okay? Because in this case, it's an external
[71:21] part that is sending to us the files.
[71:24] The problem, it is that they are sending twelve, fifteen, 20 files. Okay? And altogether,
[71:32] at the same time, it's trying to
[71:35] be inside the system. My system is not a cluster.
[71:39] It is dedicated,
[71:40] okay, in the server and
[71:44] are a stand alone, of course.
[71:46] Then all of these
[71:50] files, okay, are trying one
[71:53] time, two times, three times, four times. Okay?
[71:56] Meanwhile,
[71:57] another file is doing the same.
[72:00] And then at the end, I have all
[72:03] in error.
[72:06] So are those files that your users are uploading, or are the files that you're pulling from them?
[72:12] Yeah.
[72:15] Because in the same moment, I have things that are out
[72:19] outbound, k, or other projects.
[72:22] But when it's coming,
[72:24] okay, always in a row, when it is more than 10 files.
[72:29] Mhmm.
[72:32] It shouldn't be happening.
[72:34] So
[72:37] is this happening for all the partners or only some of them?
[72:40] Some of them.
[72:42] Okay.
[72:43] And is the problem on the sending out, or is the problem on the files actually arriving in? Where is the error?
[72:50] Arrived the first time. Okay. Arrived. It's an how can I say? The status of I need to do something. Okay? Mhmm.
[72:59] Arrived the file. The file has catch case. The case. Okay?
[73:04] And then when it's trying to put in the target, okay,
[73:08] then it's having the problems, and it's retrying and retrying and retrying two, three, four, five times. Okay? Yeah. And then all are sending
[73:19] Seric IDs.
[73:20] Okay. So when you're sending, are you doing send to partner with advanced routing, or are you doing basic app, or are you doing publish to account?
[73:29] Publish to account, sometimes
[73:31] have have basic
[73:34] application, other times are routes.
[73:37] Okay.
[73:38] So
[73:39] yeah. Yeah. Yeah. Yeah. So you will need to actually solve that in multiple different ways depending on how it's happening. So
[73:47] part of the
[73:50] let's see. So you If you prefer if you prefer,
[73:54] give me a a mail, okay, and I show you the last case that I have this problem because this problem, it was not existing some months ago.
[74:04] Yeah. It was one. Okay? One or two. Yeah. But not ten, twenty, or hundreds.
[74:11] So I would actually recommend you to open a ticket with support at this point. Yes. Feel free to cc me,
[74:18] or I will look it up after that,
[74:20] because
[74:22] you probably need tuning
[74:25] on the server itself,
[74:27] because there is parameters on the server configuration when we can increase the pools of things,
[74:32] but also tuning down on some of the transfer sites so we don't try to send that many files at the same time in the same direction.
[74:39] That's why I was asking what kind of files these are.
[74:42] So the way ST works is with the queue, and the problem is when you this queue gets too big.
[74:47] We're trying to self protect, but, occasionally, if the queues are not configured properly,
[74:52] you end up with this kind of a situation where it basically
[74:57] gives me Gets more It's not a cluster. It's not a cluster. It doesn't it doesn't matter. Yeah. It doesn't matter if it's cluster or not. The queue is still working the same way. If anything, because it's not a cluster,
[75:11] you're in a more precarious mode because you have a single transaction manager running.
[75:16] In a cluster,
[75:18] a note note
[75:20] so the the only difference between cluster and not is that in the cluster, you'll have more TMs to actually run with the data.
[75:27] But in your case, you have all the inbounds and all the outbounds on the same server. And there are queues behind the scenes, there are pools behind the scenes. And my gut feeling here is that you just need tuning.
[75:40] But Okay. All in the high in the same highway.
[75:44] Yes. Yes. I I understand. But
[75:47] it will be multiple tuning. So one of them will be to allow you to to get your threats threats numbers and so on to get a little better, especially if this is an old server that was never tuned.
[75:59] It's possible that it always had needed. It is just that your volumes increased,
[76:03] so now it's a problem. Right?
[76:06] Mhmm. But also,
[76:08] we have some controls on specific places like transfer sites and subscriptions where we can actually
[76:15] shrink
[76:16] the number of parallels in there
[76:20] so that they don't come from the database that fast.
[76:23] So it is a question of opening the funnels a little bit more, but also reducing the number of events going at the same time so that when they they go to a time a little in a different way,
[76:36] they actually don't
[76:38] step on each other toes.
[76:39] So start with the support ticket,
[76:42] because especially now that you have a live example with log files and everything, let them look through it. Let them do some tuning,
[76:49] and I'll warn you something. This will not be, oh, this is how you fix it, that's done. It will be a long process of trying things one after another to get it better gradually
[76:59] until we get to the place where it actually is set up properly.
[77:04] Okay.
[77:06] And just maybe
[77:08] they'll see something in your patterns that tells us that there is something else going on, because it might be a bottleneck on your file system. It might be a bottleneck with the database. I know you're single server, but there is still a database running over there. We might need to do something on the OS level. You might need to it might be just give more memory on this process, something like that. So start from there.
[77:31] Okay.
[77:32] Make sense?
[77:34] Thank you very much.
[77:36] Okay.
[77:37] Jeff, unfortunately,
[77:39] no file transfer templates. Technically, there is there is something called like that, but it's only for two kinds of sites,
[77:47] Connect Direct and old
[77:50] I don't even remember the name of that one. File something or another that we built years ago that is never used.
[77:57] Okay. I'm sorry.
[78:01] Question from Ed in the chat for everyone on the call. If someone is using j JMeter,
[78:07] please talk to Ed if you're using JMeter,
[78:09] especially about HTTPS.
[78:12] Okay.
[78:13] Sandbox.
[78:15] Jeff, that's a question from you again. So the sandbox
[78:19] is
[78:20] a safe area
[78:22] where we copy a file
[78:24] before we start processing it. So if you're using advanced routing and you go into advanced routing for every route
[78:32] that
[78:33] matches,
[78:34] we'll start the sandbox. Now in the newer versions, if all the steps in the route are only sent to partner and published to account, we will
[78:43] not create the sandbox.
[78:45] However, the point is
[78:47] the file gets copied from the sun from the subscription folder, get put into the sandbox. We do all the transformations
[78:54] in the Sandbox,
[78:56] and then we send the file out from there or publish it or whatever.
[79:00] That way, if someone deletes the subscription folder
[79:03] copy or someone changes it, we still have the original file and we're working on it.
[79:09] That's all about the sandbox. It's just the safe area we're working in.
[79:16] Make sense?
[79:21] Sorry. Yes, It
[79:23] makes perfect sense to me. Thank you.
[79:26] One quick question. So
[79:28] so once the file is copied from the subscription to the sandbox,
[79:32] transformations are done, and then the file is routed successfully.
[79:38] Or if it fails,
[79:39] is the file then deleted
[79:41] is that
[79:43] file deleted from the sandbox?
[79:45] As soon as you exit the route, we delete the sandbox. It doesn't matter how it exits.
[79:51] The exit of from the route deletes the sandbox.
[79:55] So if you want this file, the transformed file to be usable, you need to do publish to account inside and back to the subscription folder.
[80:03] Or what we talked about at the beginning of the call with Edig, if you do an archive for success on error, you'll have it in the archive for us. The sandbox is destroyed as the moment when
[80:15] the route finishes,
[80:16] regardless of how it finishes.
[80:19] Perfect. Thank you, Annie. Yep. You don't need to worry about those. And the sandbox is usually in the home folder of the user.
[80:27] However,
[80:29] you can also move it to a separate location,
[80:32] which is local to the server as opposed to being on the shared storage.
[80:36] This is coming with its own set of issues and its own set of warnings, but that's an option.
[80:43] But the sandbox is invisible for end users. Your OS admin can see it, but no one else.
[80:49] And it's essentially destroyed
[80:51] as soon as the route is out. So you didn't don't need to worry about it.
[80:56] Perfect. I understand
[80:57] perfectly. Thank you, Annie. Okay.
[81:00] Okay. I'm seeing a lot of reports about the database that cannot be stopped unless you do stop d b with the force execute.
[81:08] I think this is what they finished with it,
[81:11] and I keep forgetting about that. So I'll take a note and go check some things again.
[81:17] That might be what you need to do. I think we're just locking it because there is live connections from the other notes.
[81:24] Yeah. Yeah. That was that was my Ed's question.
[81:28] Basically, what what's the procedure if you wanna stop
[81:31] primary?
[81:32] I guess, stop secondary, stop primary, then stop start secondary back up?
[81:37] That's the cleanest way, but you also can do stop d b with an f
[81:41] at the end that should force the d b to go down.
[81:46] Okay. Yeah. Didn't know if that's gonna
[81:49] happen in case for secondary. It won't, but it will be longer. So if you can take the outage, I'll do stop secondary, stop primary, start secondary just to clean it up. It will probably take shorter time than the stop d b will be. Because what the stop the stop DB with the force
[82:06] will essentially
[82:07] send the force close to the secondary,
[82:10] and then the secondary will need to reconnect to its to its own database anyway.
[82:15] So there is a wait time over there, so you'll still have a couple of seconds, minutes, whatever outage over there while it's switching to the other database.
[82:23] On the other hand, if you don't stop so
[82:27] I
[82:29] I need to do some more reading because I think that you should be able to stop the secondary without stopping its database so that when it comes up after that,
[82:40] the database is already live. Although this database don't it it on especially on the edges, it doesn't take that long to start. So stop all should be just fine on the secondary.
[82:51] Got it.
[82:54] Yep.
[82:55] And to note for Mirza from
[82:57] the Chris again, reminder, don't forget you have server locks on the edges. You'll need to do the same thing on the edges
[83:04] and so on.
[83:06] So I see a couple more questions. We are five minutes to the hour, to the end, so we'll just try to cover as much as we can.
[83:14] Nishan,
[83:15] when user writes the files on ST, the file gets consumed before the user has finished writing. How do we solve this?
[83:22] What protocol?
[83:27] It is the SFTP protocol.
[83:29] SFTP?
[83:31] Okay.
[83:32] If it is SFTP so okay. Someone will need to take a look at your scenarios because SFTP,
[83:39] the way the protocol works is that
[83:41] so do you have a folder monitoring the folder from where you are consuming?
[83:48] No. The the reverse part is working fine. We have file mask
[83:51] in place so that we don't consume in in process files. But when user is pushing the file on secure transport using the credentials which we have shared with them Mhmm. In that case, the file gets consumed before they even write the file.
[84:05] That shouldn't be processed. We
[84:07] Yeah. Yes.
[84:09] It is happening especially
[84:12] for the bigger files. When they're writing the files halfway
[84:15] down, the file gets picked up,
[84:17] and then it fails because
[84:19] the format is not right.
[84:22] Yeah. So okay. So with SFTP,
[84:25] the way the protocol works
[84:27] is that
[84:29] at the end of the file, you need the end of file character
[84:33] to tell us we got the whole file, and we will not trigger anything until we get it.
[84:38] So
[84:39] check with your partners because it's I'm hearing big files. So check off with your partners if they're using WinSCP
[84:45] or something similar that splits the file into pieces before sending.
[84:51] Because if this is happening,
[84:53] that's where the where the problem is.
[84:57] So for from an ST perspective, we will not consume the file until we get end the file.
[85:03] And if the client sends us end the file for a partial file, we cannot recognize that this is a partial. Now there is a change coming,
[85:11] actually,
[85:13] in the product that will allow us to do something about
[85:22] Okay. Okay. I got it.
[85:24] So thank you so much.
[85:42] Anna?
[85:47] Annie, I think we lost you a little bit.
[85:55] She's probably trying to get back on. So
[85:58] two minutes to go, everybody,
[86:02] in case she needed to finish up that
[86:05] topic.
[86:06] Alright. Thank you, everyone. Your questions have been insightful.
[86:10] I need to drop to get on a call with the client. Have a great day. No problem. Thank you very much.
[86:17] We'll probably
[86:18] whoop.
[86:19] Yeah.
[86:20] I think we'll probably have to
[86:23] end this
[86:24] here.
[86:26] I think Annie is
[86:30] probably lost
[86:32] connection here. So thank you all so much for attending,
[86:35] and
[86:36] we appreciate it. February 6 is the next ask
[86:40] Annie. So
[86:41] we
[86:42] look forward to seeing you on the next one.
[86:45] And I'm back. I'm sorry. I have decided
[86:48] that we need to finish the meeting earlier.
[86:51] I
[86:53] apologize for that. So,
[86:55] I don't know where you lost.
[86:58] Yeah. And if we got the last question,
[87:02] but what I was saying was with SFTP,
[87:05] you have a client sending EOF when they shouldn't.
[87:10] So okay.
[87:12] So and we're on time anyway.
[87:14] So thanks everyone for joining. Sorry, Lucy, for cutting you off.
[87:18] As Lucy said, next one is February
[87:21] 6,
[87:22] I think, and we're going to try to keep that on the first Thursday of the month unless it's too too early like it was this month.
[87:30] So
[87:31] thanks everyone for joining. Do let us know if you have any additional questions.
[87:36] Don't forget community portal that has ideas
[87:39] and everything else, but most importantly, you can just ask a question over there.
[87:43] And thanks again to everyone that assisted with answers on the chat.
[87:50] And happy new year, everyone. And
[87:53] hopefully, everyone have a very successful '25.
[87:57] Right. Happy new year. Thank you, Annie. Happy new year to you. Everybody. Thank you. Yep.
[88:03] Bye bye. Bye.
[88:05] Bye.