ask-annie/out/799869428/transcript.txt

# Transcript: 799869428
# URL: https://vimeo.com/799869428
# Duration: 6771s (112.9 min)

[0:05] Annie, I can start. This is Kim. I'm from Invesco,
[0:09] and I manage, secure transport,
[0:11] here. We have, LEC cluster in both our nonprod and production.
[0:17] So my question
[0:19] is I would want more of your opinion, Annie, on this is that
[0:24] we did, from the marketplace,
[0:27] download the transfer site for s three,
[0:30] And we have been using it
[0:33] for our
[0:34] company owned AWS
[0:37] environment.
[0:38] And we have been quite successful with it using with the
[0:43] users who they own the s three bucket.
[0:47] So they have a user assigned, and they are able to prop provide us with the access key and the secret key.
[0:54] However, the company is moving now more into
[0:58] automating
[0:59] the internal process for an s three bucket.
[1:02] And instead of assigning users
[1:05] to, say, an application team,
[1:07] they're now going to have roles.
[1:09] So with the roles, you know, there's not a secret key or access key.
[1:15] The permissioning is now,
[1:17] different.
[1:19] And so I am going to start to try testing with, you know, an ARN connection,
[1:25] and I
[1:26] saw a KB article on the support site about that and the permissions that need to be assigned and I think also using a profile.
[1:34] But
[1:35] how many people are using that type of connection
[1:40] to an s three bucket? I'm asking because I wanna understand, is this gonna be something that, you know, I'll be needing to do from scratch, or does support have
[1:50] experience with that?
[1:52] Because we are also getting a request to potentially
[1:56] transfer data externally to an external customer's
[2:00] cloud environment.
[2:05] Well,
[2:06] part of the disadvantage
[2:10] Just to update, I'd like Prakash
[2:12] here from ETS. We are using air and base to access to the f three bucket.
[2:17] We are able to transfer the files, lot of files, like, around 100 plus s three bucket we are routing.
[2:23] Just year end and the policy needs to be updated for that year end
[2:28] and the s three buckets proper. If you are using the KMS keys,
[2:33] your EARN should have the KMS key policy allotted,
[2:36] and the s three bucket KMS key also allotted.
[2:39] So that these two values you need to make sure.
[2:43] Okay. Correct. Do you mind if I since you have implemented it, do you mind if I contact you outside of this user group?
[2:51] I'm not sure.
[2:52] Okay. That's fine.
[2:56] Our understanding too with roles as we look at just implementing from the beginning
[3:01] is roles are more complicated if you're using an on prem system
[3:05] because you got extra token authentication that you have to deal with.
[3:10] Whereas roles are much more intuitive if it's hosted in the cloud to begin with from your system talking to the s three bucket.
[3:20] You have to deal with chaining and other things.
[3:34] Okay.
[3:35] So what I started saying was that I'm at a disadvantage a little bit because I only hear about problems about these three sites. When people are using it successfully, like the two guys they just mentioned,
[3:49] I don't get involved.
[3:52] So,
[3:54] you know
[3:56] So come.
[3:58] Do you have anything else on that, or did this help?
[4:03] This helped that I'm not alone.
[4:05] Support
[4:07] should have some experience with it. I just had one just follow-up if Mhmm.
[4:13] Has anyone started transferring
[4:16] to an external partner's cloud environment?
[4:23] What does that mean? So on prem on prem
[4:27] at the
[4:30] yeah. Go ahead.
[4:37] I cut off someone.
[4:44] Sorry. She said the external vendor. Right? Like, any external cloud means. Like, AWS,
[4:49] she she is referring or, like, a
[4:53] Oh, okay. So, like, my company,
[4:55] we're going to
[4:57] be an AWS shop. Right? But, say, I may have an external company that's
[5:03] they can be AWS or another cloud
[5:06] provider.
[5:07] I was just asking if anyone on the call has experience
[5:12] sending
[5:12] transferring to an external partner
[5:15] cloud environment. I'm on prem. I guess I should add that, Annie.
[5:22] I just wanna know if anyone has been doing it.
[5:26] But our company, we are in the AWS. We are sending to the external vendor with the AWS accounts. They have different accounts. Right? So we are able to transfer the files to their s three buckets.
[5:40] Okay. Thank you.
[5:42] So
[5:47] part
[5:48] of the problem is that when you go against an s three, you don't always well, you always know if it is AWS
[5:55] or Azure or whatever.
[5:58] But it might not always know if it is on prem or really up in the AWS.
[6:03] Right? So
[6:05] Yeah. That's right. Thank you.
[6:08] K.
[6:09] And,
[6:12] you know, I like to push just like that when I don't almost don't need to talk. So come
[6:17] what you might want to do is to start the community post,
[6:22] you know,
[6:23] so that other people may chime
[6:26] in with ideas.
[6:28] Or do some tries and then
[6:30] post where where you are.
[6:33] Our support team will be happy to help, but
[6:36] in a little in in some ways, they're where I am with that. Unless someone has a trouble, they no one talks to support about that.
[6:45] Alright? I'll yeah. I'll take that suggestion. Sometimes
[6:50] yeah. Sometimes we might not know if people are using something
[6:55] until
[6:56] we have or someone complains about it.
[7:00] And that's not just the new features.
[7:02] Right?
[7:04] Okay.
[7:08] Anything else on that one, Karam?
[7:11] Yes. About cloud, I have a question, if I may intervene.
[7:16] Sure.
[7:17] For Absolutely.
[7:20] Cloud based installation is, the same, that on prem. The nothing really changed.
[7:30] For the product itself,
[7:33] correct.
[7:34] Are you talking about your own cloud or Appsway cloud?
[7:40] Sorry. Can you repeat, please? Alexis?
[7:43] Yes. Can you can repeat about
[7:46] yeah. Your cloud or
[7:49] cloud?
[7:50] No. No. This is about private cloud.
[7:54] Okay. So on private cloud, absolutely, it's the same.
[7:57] The reason and I'm asking is in actual cloud, it is the same again, but there are some rules on top of ST
[8:04] that our managed cloud applies.
[8:07] But for private cloud, at the moment, the software and the rules, everything is absolutely the same as on prem.
[8:15] Same software,
[8:16] same installer,
[8:17] same same policies.
[8:19] Okay. So,
[8:20] I I mean, the last person, asked about, file transfers. So it's basically the same thing. Nothing really changed.
[8:28] Okay. Fine. Thanks.
[8:31] Perfect. Thanks. When
[8:35] you say it's absolutely the same, let me The change where your data is.
[8:40] Yeah. The changes will be where your
[8:44] yeah. Go ahead.
[8:48] My question was going to be with regard to
[8:51] admin tool sets.
[8:54] So
[8:55] for example, I get I can't tell you how many requests a week that want information from the system that you just cannot get any other way other than going to the database and running your own custom queries.
[9:07] The APIs just don't have enough
[9:10] comprehensive and configurations to get you what you need.
[9:14] So in a in a cloud environment, managed or private,
[9:18] would my tool set
[9:20] capabilities be the same? Or by moving into the cloud,
[9:24] is that now restricted in a different fashion to where things that are available to admins have somehow been limited?
[9:34] So that will depend on the access that your cloud is giving you.
[9:40] If you still have OS access, they d t DB access, you'll have the same stuff.
[9:45] And you know what I'll say about the DB.
[9:48] But
[9:49] the product is the same, and we when you up when you put it in the cloud from ST perspective, from the administration perspective,
[9:58] we don't apply any additional or different
[10:01] changes
[10:03] in j n r
[10:05] change, literally.
[10:08] So it really depends on your environment. If you're a heavy user of database, curious,
[10:13] you will need to check with your provider for the database in the cloud to see what access they will give you.
[10:21] And, you know, the usual warning of please don't go against the database because it can change. I need to say that, but I also understand where you are going with that.
[10:32] Can you share what kind of queries you are running against the database?
[10:38] When we were
[10:39] when the Oracle was the back end database, my tools were
[10:43] things like Toad,
[10:45] you know, as a as a DBA type of a tool. Mhmm. The current installation,
[10:51] which is also an LEC, is running
[10:53] Microsoft SQL Server. So I'm just using their standard enterprise manager in in query tools. And it's only read only. I mean, I'm not running any updates and things, but I just when I need to get information,
[11:06] that is something that I'm much more
[11:09] easily capable of doing than trying to reverse engineer
[11:14] four or five different
[11:16] API API calls if I can get them to get what I need. You know? Yeah. But what kind of queries are you running? Are you doing
[11:25] are you checking for quizzes?
[11:27] Yeah.
[11:28] But yeah. I'm sorry.
[11:32] A lot of the queries have to do with
[11:36] dealing with
[11:37] cleanup. So I don't know about anyone else's organization, but one of the challenges that we face is that we get work work tickets all the time for the provisioning of onboarding and new configurations.
[11:48] And I get barely virtually nothing
[11:52] when a relationship ends.
[11:54] So over time,
[11:56] we find we have
[11:58] detritus and objects, configuration objects out there in our environment that are just they're dead. They're not being used.
[12:05] And so a lot of the things that I end up doing are running
[12:09] utility admin sorts of queries to identify
[12:12] things that are used, things that are not used,
[12:15] things that aren't even configured anymore. You know? So I'll run a query to see if I've got transfer sites that are unassigned,
[12:22] you know, to a subscription and and things of that nature. Or business will come in and they'll ask
[12:28] how many different configurations
[12:29] go to a particular host or how many versions of this external partner's
[12:35] host information do you have? Because over time, they may have modified
[12:39] and moved into three or four different environments or changed
[12:43] network providers, etcetera. So it's a lot of things along that line that I'm constantly
[12:48] running up against or,
[12:50] you know, configurations
[12:52] that are based on
[12:54] certificate authentication versus password
[12:57] to the same partner. You're you're cleaning that up. So a lot of things along that lines,
[13:03] where
[13:04] you just don't have that capability in a in a standard API. You have to string them together in in most cases to figure it out.
[13:12] Okay.
[13:13] The reason I am asking is because r and d are always interested in what people are doing. So because they're looking for what else need APIs for.
[13:23] So Right. If if you haven't been doing that,
[13:27] we have the ideas portal, as you know. So if something cannot be done through the API or is very, very,
[13:34] let's call it, cumbersome,
[13:37] just post over there. Let them know what you're looking at.
[13:41] Because for some of those things,
[13:43] it might they might be able to throw a new API together,
[13:47] but they cannot invent something if people don't ask for it. So just Sure. The stuff you are talking about, I usually don't even go against the database for that kind of stuff. I'll do an XML export
[13:59] and use my old XML tools,
[14:02] which are even clunkier.
[14:04] But,
[14:05] yeah, I understand.
[14:08] Yeah. The API is not very good with some of those things.
[14:11] Are you using the new API two zero or the one zero four when you tried the APIs? It's it's it's two point zero. Yeah. Okay.
[14:20] Yeah. Because And I noticed that they've consolidated quite a few of them. It used to be that there were you know, take a half a dozen of the one dot four APIs, and now they've done combinations,
[14:30] especially in things with advanced routing. But even with advanced routing, we've done some some centralization and created,
[14:38] for example, generic account
[14:40] that's public. All the transfer sites are public so that we get inheritance and reuse of those transfer site objects for all the others.
[14:47] And so when you go against a configuration looking to see what's related,
[14:52] there's not a direct relationship all the time between
[14:55] the separate accounts. And so you have to sort of come up with makeshift workarounds for that, which is what I've done, you know, within regard to the advanced route and the database, but I don't see that,
[15:07] yet in, APIs.
[15:09] Yeah. And there that's always a good use case. And as I said, drop some of those into ideas.
[15:16] Even if they never implement them, that at least
[15:19] or not immediately, that at least will give them some idea for the next generation of APIs.
[15:26] Will do. Thank you. Bake it in ST or outside of ST or you know?
[15:31] Sometimes,
[15:32] it might be as easy as people asking about it. There is way too many things. We have a huge database, a huge configuration.
[15:40] Building KPIs on top of it is not trivial. Right? So okay.
[15:45] But,
[15:47] we I I'm seeing Steve. We he has his hands up.
[15:51] Yeah. Hi. So,
[15:53] I just have a two questions, actually. The first question is, oh, I'm running an enterprise cluster,
[15:59] and I'm running with, 24 gigs of RAM and
[16:03] four CPUs.
[16:05] And my CPUs are
[16:07] are pegging. I mean, almost 200%.
[16:10] And
[16:11] question, has anybody ever had that situation?
[16:16] It's
[16:17] half normal,
[16:18] but I also would add some more power to those machines. It depends on what CPUs you're using, obviously.
[16:25] But Okay.
[16:27] I bumped up a little bit. Think of that as a a big server.
[16:31] And 24 gigabytes, you are running
[16:34] very close to what I would call a minimum these days.
[16:38] Okay.
[16:40] I I these days, I'm seeing transaction managers running with eight or 12 gigabytes of memory already.
[16:47] So it depends on your traffic. And I don't know your environment. I don't know how much traffic you have through it. But if you are using more than, say, three protocols
[16:57] and if you have a lot of
[16:59] transfers at the same time, I'll look at some better hardware than that. You know, just bump up the hardware if possible. If you're not on virtual, obviously, it's harder.
[17:09] Yeah. It's virtual.
[17:11] Yeah. The the CPU yeah. Go ahead. Sorry. No. I was just saying it's they're virtual servers.
[17:18] CPU staying high doesn't worry me that much. It's just
[17:23] Java being Java sometimes.
[17:26] Although it's always that high,
[17:29] I would start. I would actually talk with our support team to see you might be in a leaky situation where something is leaky. Which update are you on on five five? Or are you on 05/05 yet? Yeah. I'm on 05/05.
[17:42] Do you know which update?
[17:44] Would be July
[17:46] 2022.
[17:48] Okay. So that's about six months old now.
[17:52] Check with support to see if they know something in the July that might be causing that. We have been having historically
[17:58] some cases where
[18:00] a preview one of the updates is just higher on CPU because of what you are doing.
[18:07] It will also depend on your scenarios. If you have a lot of CPU heavy operations,
[18:14] it might also stay high. But if it stays that high twenty four seven
[18:19] and always training all the way up Very different. I would start trying to figure out what's going on. It be
[18:26] it it might spike there.
[18:29] And when you're very busy, it might stay there for hours sometimes,
[18:33] but it always stays there.
[18:35] Well, I see it fluctuating.
[18:37] It'll go up to, like, a 180%,
[18:39] then it'll go down to a 100%.
[18:41] Then it'll go back up to a 170,
[18:43] and then down to 80 or 90, and then back up. I mean, just seems like it's constant.
[18:49] Yeah.
[18:50] It it sounds a little your machine sounds a little underpowered for what you are doing, would be my first guess. Okay. But again
[18:58] Yep. Go ahead. Mhmm.
[19:00] The check with support. Open the ticket with support with your update built and see with their proposal.
[19:05] You need to update anything now anyway. I mean, six months Yeah. Ago. It's about time to update anyway.
[19:12] I'm doing it today on our development environment. So we'll be moving that up through our our systems today,
[19:19] and then probably in about three weeks, we'll have it all done.
[19:22] I I do have a I yeah. We're doing the January 1. Okay. I do have a case open with support. I've had it open since October.
[19:30] Oh,
[19:32] please keep them again.
[19:34] See what's best for with them. So they keep seem seems like they keep asking for the same information and telling me the same things. But
[19:43] I'll keep continue to work with them.
[19:46] So
[19:46] part of the problem with this kind of issues is that it's not like, oh, you missed the checkbox here.
[19:52] It's usually a combination of who we have to take. And sometimes
[19:57] one of the times, believe it or not, it was the storage the way the storage was mounted that was causing the trouble. I still don't understand how that was happening, but Okay. Know?
[20:07] Yeah. We're running external storage to an Oracle back end as well. So,
[20:12] you know, that's possibly maybe an issue too. Anything. Yeah. Yeah. Alright. I'll I'll just continue. My next question, when is the next version coming out?
[20:23] Usually,
[20:25] the last Thursday
[20:27] of the month
[20:28] unless No. No.
[20:30] The next version of, like, five, six or whatever you're gonna be moving to.
[20:35] No one knows yet. We have Oh, you don't have So we we are officially in what we call continuous development.
[20:42] Okay. So there is no plans whatsoever to jump for a chance until they have a new plan in place. She's just like the current idea.
[20:51] Guys, if you're not talking, mute yourself, please. Okay.
[20:55] That's fine. Yeah.
[20:58] Yeah. She's they're giving her some tough questions, it sounds like, right now.
[21:03] We got people from UPS.
[21:05] Tony, we can hear you.
[21:07] Thank you.
[21:09] I jumped and closed all the mics. Okay.
[21:13] Sorry about that. Okay. It doesn't shut it doesn't close yours, so I'm glad I can test a bit also. And I, used the opportunity to tell you, Adi, that we received some questions in the the chat. I saw that. Good. Okay. So, Steve,
[21:31] if they decide to change the rules, we'll hear all about that. The current plan is that it will be five five with updates
[21:40] continuously,
[21:41] probably forever or until we change the technology radically.
[21:45] Okay. So Thank you. Thank you. On top of it, though. Okay. I do. Yep. I have five people with hands up, and they have questions in chat. So let's go to the voice, and then I'll go to chat. Who is next is
[21:57] hold on. I have numbers here.
[22:00] Kam?
[22:03] K. I just had a question.
[22:06] We are looking to implement the file maintenance application. We're on five five.
[22:11] Yep. And I was just wondering,
[22:13] if anybody had experience with that. The reason and I'll need to probably do a slow,
[22:19] implementation of this because our storage by our users have just been unchecked for several years. And I just wanted to know if there is, a best practice or a or even if anybody has experience with it that they wanna talk about on if they're using this, when they first started using it, what were some any gotchas that I should be aware of?
[22:41] Before anyone else jumps in, take it very, very slow,
[22:45] either by business unit or if you have big users one by one and them and so on. Because otherwise, you'll over helm the process overnight that is deleting the files.
[22:57] So that's my only advice on that.
[22:59] Okay. So split
[23:01] the users, you know, split it to concur until everyone is set up properly.
[23:09] Anyone else to anything have any sorry. Have anything to add here?
[23:19] Nope. Okay.
[23:22] Rao.
[23:23] Yeah.
[23:26] So,
[23:27] I mean, basically, we are running two enrollments of both the test and production.
[23:32] Yep. So starting
[23:34] from about one, one and a half months, especially
[23:38] almost every week or
[23:41] every two weeks,
[23:42] we are finding that
[23:45] the
[23:45] SD cluster is
[23:48] out of sync.
[23:50] I mean, I'm not sure whether they're out of sync. Basically, like, I have two nodes, one and two.
[23:57] When I log in to one and check the cluster management page, I see that node two is offline.
[24:04] When I go to log when I log in to node
[24:07] two and go to cluster management page, I see that one is offline.
[24:13] We have opened a case with the actual support. We have been working for almost one one and a half month.
[24:20] And this has still has some performance tuning and all.
[24:23] But even even after
[24:26] doing all these things, we are still having this problem.
[24:29] So last occurrence was yesterday.
[24:33] Not sure this is a right forum, but,
[24:36] I mean, I just wanted to check with you,
[24:39] like, where
[24:40] we need to check-in
[24:42] order to resolve this issue permanently. Okay.
[24:45] And you say it's happening every few weeks?
[24:48] Yes. At least at least every week or
[24:51] alternate weeks.
[24:52] Okay.
[24:54] What I would start, quite honestly, is talk to your networking team and your DNS teams and see if they are not doing some reset about the same time.
[25:05] So Sorry. Can you repeat, please? Sorry.
[25:08] Check with your networking team and your DNS teams in the environment.
[25:13] This communication
[25:14] is basically between two nodes,
[25:17] and it's very successful
[25:20] if they do a networking
[25:22] reset
[25:25] or do networking,
[25:27] maintenance.
[25:28] ST might get a little grumpy about that, and sometimes a machine doesn't
[25:33] send things in.
[25:35] But
[25:37] unless
[25:39] because it's happening
[25:41] after some time,
[25:43] I believe you have a leak somewhere.
[25:46] Maybe the OS lab, maybe the kernel, maybe ST itself,
[25:50] or maybe the the DNS adapter you're using. Who knows?
[25:54] But my good feeling is that it's outside of ST or at least not entirely ST. They if it says it cannot see, that means that they literally cannot reach it or it's timing out.
[26:14] That's the best I can tell you, I'm afraid. I mean, it's one of those cases where the they will will be in the details.
[26:21] Okay.
[26:22] But starting one node is showing
[26:25] both nodes are up
[26:27] Yeah. Restarting. Yeah.
[26:29] And that's why that's why I'm thinking there is something in the caching of the DNS, in the caching of the networking
[26:35] that either is leaking slowly so it gets over helmed.
[26:39] So at one point, it just doesn't see it anymore.
[26:43] K. Or
[26:44] there is a change that doesn't get repopulated
[26:48] into ST, so even though ST was okay. When you restart,
[26:52] remember that we are refreshing the memory from scratch. We're doing all the caches from scratch. We're doing all the networking connections from scratch. Right?
[26:59] One of those things that are set of one things is fixing it. So your machines are okay, but something in them is not getting updated on time.
[27:09] Okay. And that's why, in my experience, it usually will be on the networking layer.
[27:15] You know, networking conductor not recognizing
[27:18] something or timing cloud. Also,
[27:21] if you don't have the IPs of the servers in the host file on both sides, you might want to add them so that you remove the DNS from the reverse
[27:31] DNS and the DNS itself.
[27:34] That was a that was a one
[27:37] recommendation
[27:38] from access support last week. Mhmm. We made that change.
[27:42] We made the change on Friday and Monday morning we found it.
[27:47] Same issue.
[27:48] Yeah. Make sure that it actually stakes them. You know, when you set up the configuration, you can tell it to use DNS as preference.
[27:55] But if that doesn't help,
[27:58] I'll start looking at DNS caches.
[28:00] And,
[28:01] also,
[28:02] it might be useful to put a networking snip for,
[28:07] but because you can't you don't know when it will happen, there will be a lot of data in there.
[28:12] Okay. Because there are so there are two reasons for this to happen. Either the connection cannot be made at all or it tanks out.
[28:19] Right?
[28:21] Knowing which it is is easier to troubleshoot, but catching which it is
[28:26] not that trivial.
[28:29] So
[28:30] work with your networking team. That's the best I can say, which doesn't mean ST doesn't have a problem. Which update are you on?
[28:38] We are with April
[28:40] 2022.
[28:44] Update and see if that will help.
[28:46] Yeah. And I hate saying that, but on the other hand, we have quite a lot of customers with enterprise clusters that are okay.
[28:55] This is enterprise cluster. We are using enterprise cluster. Otherwise,
[28:59] you you don't have this kind of a page over there. That's why I didn't even ask. But what I'm saying is we have a lot of customers that don't report that issue.
[29:08] So it's something environmental.
[29:10] I would update and see if do you see that both in your development and production?
[29:16] Definitely. I mean, it it occurred once in production
[29:19] on
[29:20] December 24, I think. Mhmm. But since then, no issues.
[29:25] And we are Mhmm. We we are running with the same version and
[29:28] the patch release
[29:30] since
[29:31] April
[29:33] 2022.
[29:34] Yeah. Since that since that time, we are using the same. We don't have I mean, we never faced this issue until 2022,
[29:43] especially for past one month of month, we are having this problem by currents. With saying that, I go to your networking team and ask them what they did in December.
[29:53] Did they update something somewhere?
[29:55] Did you had an OS patch on those servers?
[29:59] Did they update the networking adapters? Did they update some networking hardware somewhere between the two servers? You see where I'm going with that?
[30:08] If something starts happening so you'll be running successfully for six months without troubles and then it started happening,
[30:15] it's on the software.
[30:16] Right? We don't have a timeline saying, oh, six months. Let's let's break it.
[30:21] But they you it it might be a security fix somewhere.
[30:25] Your security team and networking team might have applied to security t a security fix on
[30:31] OS kernel, for example, which is causing the networking adapter to misfire a bit a bit. You know?
[30:39] ST
[30:40] is not working in isolation in this kind of things.
[30:43] So
[30:44] start from there. You know when it happened first? So go about two weeks be back from that and start asking questions internally. Who did work around those for servers and the communication between them?
[30:58] Okay. That's the best thing you can do at the moment, I think.
[31:03] Thank you.
[31:05] Okay.
[31:07] Let me see who is next.
[31:10] Mishra.
[31:11] I cannot pronounce names. I apologize.
[31:15] Oh, not a problem. So
[31:18] so,
[31:19] actually, we have a unique situation where where we have some partners and they are using, you know, end user APIs to send us files. That was a poor setup. Don't ask, like, how they came up. It's
[31:30] it's how it is. And now the security team is coming back to us, and they say, hey. We are not going to allow you to use basic authentication.
[31:36] Turn it into some kind of token based authentication. This is just, like, using something like Okta
[31:41] Okta or something like that. And I know there is a plug in available on the marketplace, but that's not what. It's just, you know, that's just a plug in. Right? You know, to authenticate to some third party,
[31:51] you know, kind of IDP service. So I was wondering, is there any solution that we can think of or, you know, something that might be coming in future out of the box in X-ray where we can just, you know, put some
[32:06] API gateway kind of thing?
[32:08] What protocol what protocol are they using? HTTPS?
[32:11] Yeah. Of course. Yeah. Okay.
[32:14] So one of the things they are now looking into is expanding the e the o out tool.
[32:21] O o o
[32:23] a
[32:24] u t h tool, o out.
[32:28] Uh-huh. We already have a plugin for that, but it's authentication
[32:31] only, not authorization. So it still doesn't work with tokens.
[32:36] They are looking into that, actually. So there might be some news. In the meanwhile,
[32:43] the best thing you might be able to do is to go to certificates,
[32:47] but even that is basic is basically certificate authentication.
[32:50] So Yeah. And also, like, that is a mess to manage. Right? Yeah. Well,
[32:55] yes and no.
[32:57] You know? Yeah. And Do something. But yes. It it
[33:01] Most most of the users that we have, like, they are, like, using automated system. So, like, maybe we have to guide them in order to use certificate, but it it still falls into the category of basic authentication. Right? It's it's Yes. Smart. Yeah. So yeah. It's still not solvable that way.
[33:17] It's issue. So the cleanest way will be to move to SAML with an external IDP.
[33:23] And, of course, I understand why you wouldn't want to do something like that.
[33:27] But
[33:28] those are the current options, and HTTPS
[33:31] is a protocol with a little
[33:34] temperamental,
[33:36] you know. On the other hand, it's also the most open of them all.
[33:40] So
[33:41] you can always write your own plugin doing whatever you want it to do.
[33:46] Mhmm.
[33:47] Other from that,
[33:49] I don't know about anything else coming down besides the o o out
[33:56] tool changes.
[33:58] So and part of them are exactly because people don't want to use basic auth.
[34:04] So we shall see what happens next. Keep an eye on the portal
[34:09] and
[34:11] posting ideas, please, or there might already be an idea over there. Go and vote or comment.
[34:18] R and D need to see that more people require the removal of the basic auth from any tank.
[34:24] Because I the problem is that when I go to them with a single customer,
[34:30] they cannot see that everyone needs it. Right?
[34:34] Part of the idea where why we have ideas
[34:37] is
[34:37] to allow people to actually tell us what is important to them at the moment.
[34:43] Yeah. We we have, you know, added into ideas, but, you know Yeah. It it it, you know,
[34:48] you know how it works. It's it's not the most efficient way to get things done, but
[34:53] I had, like not, But it's pretty much the only one at the moment. You know? An idea actually have more power to get that and get to something than me going to telling them. Just saying.
[35:05] Mhmm. Okay. Go ahead. You you have another question?
[35:09] Yeah. So, like, I had two more. One is, like, not for the audience, but I was wondering where the where are we with Ada?
[35:18] Getting there.
[35:21] Wait for news later in the year or early next year. I don't I don't have a date line yet, but they are working on it won't be called called HADA anymore.
[35:30] So this is basically high availability
[35:32] across domains, guys, for the ones that don't know what we called HADA back in the days.
[35:37] Reach out to your account executive if you're interested about it so he can pass the information. So when they're ready for beta phases they were looking for, they'll be working for comp with some companies.
[35:48] So not there yet. They're working on options.
[35:52] Okay.
[35:53] And I know you had been hearing that for about
[35:56] few years.
[35:58] At at least three, maybe more than that.
[36:02] Well,
[36:03] we're trying.
[36:04] As you know, it's a technological nightmare because of what we're actually doing.
[36:09] So I understand. We hope to have something around
[36:13] that at some point. It might not be exactly what we plan to do. It might not be
[36:19] fully available. It will require some of the additional suits.
[36:23] But some of those,
[36:24] developments going on outside of ST and into the MFT space are to support something similar.
[36:30] But if you're interested to talk to your account executive so he can send you down his r and d team so they know you are coming.
[36:38] So
[36:39] makes sense?
[36:40] Yep. Yep. Yep. Thank you so much. Okay.
[36:44] Thomas
[36:46] how do you pronounce your last name, Thomas? Ronneby?
[36:49] Do you pronounce the g there?
[36:53] I don't know. Yes. I do.
[36:56] Okay. You got close enough. I knew who you were talking to.
[37:00] Sorry. My question is on is you can mhmm. I'm sorry. Go ahead.
[37:04] No. I was going to say, sorry. Too many languages, everyone pronounces different letters. Mhmm. I'm trying.
[37:10] Yeah. It's alright.
[37:13] The my question kind of dovetails in with the talk about the s t five five monthly updates. I wanna know if there's a is there a best practice suggested
[37:21] on how often we should be applying them?
[37:23] You know, getting an outage once a month is going to be
[37:27] a little bit more difficult. But, you know, if that's the nest what y'all suggest, then that's what we can go with. But
[37:33] No. If you ask r n d, the answer is put it every month, which is, as we all know, not going to happen.
[37:41] My advice is plan to do it at least
[37:44] two or three times a year
[37:47] so that you have the ability to just keep one of them if things go horribly wrong.
[37:52] Mhmm.
[37:54] I would make a point to try to do it quarterly.
[37:58] Okay. You know?
[37:59] That gives you and, of course, that means that if the build is really bad or if you find problems during the build, you can skip one and still be okay.
[38:08] What you never ever want to happen is to run more than a year behind because then the update becomes a little harder.
[38:16] But, also,
[38:17] each of those updates has security updates.
[38:20] So running six months or eight months or a year behind on security is going to kick you in the butt sooner or later.
[38:28] Sorry for the expression, but we all saw all those troubles with security in the last few years, and this will get worse and worse as time continues.
[38:38] And especially if you are using
[38:43] security a lot of secure transfers with other partners, you know, you're only as in a network, you're only as secure as the weakest links.
[38:51] If you become the weakest links,
[38:53] the weakest link,
[38:55] that's a huge problem for your organization.
[38:59] Understood. Thank you. So quarterly,
[39:01] but
[39:03] and this is important. Keep an eye on every release that comes out. If you see, if you spot in the release, a security update that really resonates with what your security team is telling you,
[39:15] bump it up immediately.
[39:17] Right?
[39:18] So,
[39:19] you know,
[39:20] I know that
[39:22] r and d will love to tell to do it every month, but we also know that taking the outage every month and the testing of every build every month is just not realistic.
[39:32] Understood. Thank you. So yeah. Okay.
[39:35] Okay.
[39:36] Anything else?
[39:38] No. Okay. Bart
[39:41] Bartimaer.
[39:42] The next one with the hands up. Yes. Good evening.
[39:45] Hello. Questions.
[39:48] First one is, in fact, we are using the XFB gateway for quite some time. Mhmm. And the support, of course, we are now going to migrate this year to secure transport.
[39:59] But I'm looking, in fact, for an optical migration parts from the XFB gateway to
[40:07] secure transport,
[40:08] and and I haven't found really a document that explains how to do it
[40:14] or what are the best practices
[40:16] if you may.
[40:19] Yep.
[40:21] So
[40:22] yeah.
[40:24] So
[40:25] had you been talking to our
[40:27] services
[40:28] team? I think you're in Europe. Right? Yes. In Belgium.
[40:31] Yep. That's correct. Yep. So talk to the services team or your account executive. We actually have a services run program about migrations
[40:42] from x u b to s t. They're all straightforward.
[40:47] So they have some But we have been migrating in the last
[40:50] Okay. But they In a way, it's Okay. They have some internal tools that can help with the migration, and they can also help with the architecture
[40:58] or some best practices. So that's more for professional services. It's a process.
[41:04] Yes. It's a process, and there is some pulling under it. We don't have something we can just give you simply because it's not one on one.
[41:12] But the services team,
[41:14] especially the European teams where most of our exit be as our
[41:18] had been doing that for a while. They have a process. They have a a whole thing going on that helps with stuff like that.
[41:27] Okay. Okay. Based on your scenarios,
[41:30] based on what you're doing. And I I I would love to be able to tell you here is the 25 steps to do, but that won't get you where you want to do. And part of it is because the XMP and test are so very different. Sometimes
[41:43] you actually need to revisit what you're doing and change it a little bit. So talk to them. Get to your account executive.
[41:51] It will not be a one on one migration.
[41:54] I I you know, it won't be switched the the turn the key down. Right? It's a process.
[42:01] But
[42:02] that's a process that had been going on for the last two years or so in Europe, and they had been getting more and more successful with that. How hack yours will be will depend on your scenarios.
[42:14] Okay. So you cannot, for example, give a rough estimate. It will take so
[42:19] many mandates or something to get an idea of what the cost price would be or how many mandates we should
[42:25] Talk to them. Yeah. They'll need to see your details. They'll need to see what you're doing, how many partners you have, and that will give them a rough idea of what you might be able to do.
[42:35] Because you it it's a huge difference if you have 10,000
[42:40] people that just use SSH
[42:42] versus
[42:43] even a 100 that use per seat and SSH and the FTP and the HTTP.
[42:49] You know, it's
[42:51] it's just not
[42:53] you know, I it's you need a crystal ball without any details.
[42:57] Yes. I understand that it will be a customer. I but I think our environment is quite small compared to some other other customers of you. Yeah.
[43:06] And that's why
[43:08] Yeah. Do you do you do you know if most customers make use of the professional services or most people say we do it ourselves and
[43:16] they contact the court? Or
[43:19] So
[43:21] unless someone is willing to rebuild everything almost from scratch, most people actually go with the services team.
[43:28] It might be for services to do all the work. It might be for services just to be advisers on that or Mhmm. Create to help you create a plan.
[43:38] But I would strongly advise to use all of that knowledge base over
[43:44] there that can't accumulated in the last few years
[43:48] just to give you the leg up. And, again, if you decide after what they proposed to you that you want to go and do it on your own,
[43:56] you know, more power to you. But talk to your account executive,
[44:00] get some people from presales and and
[44:03] and services. I don't know how exactly they're organized at the moment. Get someone to talk to you or to run your two things. At least have a conversation with them and see what they have.
[44:12] If you don't like what you're hearing, you know, scrap everything start from scratch, but that's usually the worst
[44:19] case scenario. Right? But yeah. But The one for professional
[44:23] services. I mean, that's why you have that service. Yes.
[44:28] Yes.
[44:29] And again, this in the last couple of years, that had been one of the main focuses in Europe.
[44:36] Get the x u v gateway people to go into ST cleanly or as clean as possible.
[44:41] Yeah. Of course, we cannot move passwords
[44:44] in some cases because they're,
[44:46] you know,
[44:47] hashed and not encrypted, and you know all that.
[44:51] But at least the heavy lifting would be doable. And, again,
[44:55] worst case scenario, have a conversation,
[44:57] see what they'll propose. They might tell you you need to start from scratch, but at least that will be based on what they see in your environment and what
[45:05] details are there as opposed to me just wearing my hands and guessing.
[45:11] Okay. Yeah. So Yeah. But I think we will have a conversation with
[45:16] with X-ray, our customer
[45:19] contact person there.
[45:21] But I also have a second question that's more for high availability.
[45:24] We are using
[45:25] secure
[45:27] transport on the win.
[45:29] Mhmm.
[45:33] Active.
[45:34] But what if you want to use the standard Windows clustering service?
[45:38] Is it something you recommend, or you say, no. You should really use the X-ray
[45:43] way of clustering and active active,
[45:46] more on the application level and not on the Windows cluster level?
[45:50] Is this Never ever
[45:53] Okay. That one is actually very easy to answer.
[45:55] Never ever use the Windows clustering.
[45:58] Okay. That's We don't support it. It doesn't work.
[46:02] Uh-huh. Our cluster is application level one based on the database behind it.
[46:07] Mhmm. Anything else is not supported will not work and will only cause you headaches.
[46:14] Okay. That's clear. Yeah. Uh-uh.
[46:17] Okay. So if if you don't have availability,
[46:20] then we need to do the
[46:22] the standard
[46:24] following the x-ray cluster from the application
[46:28] level. And then we have support. Otherwise, we don't have really support. Okay.
[46:32] You don't have support and it will not work
[46:35] because,
[46:36] you know, people have tried. In these cases, they call me to go fix it, and then need to go to explain to them that they need to start from scratch because what they have will never work.
[46:45] Okay. But that's very clear. Thank you. Yeah. So also keep in mind that even though we run on Windows, we are not a Windows native application.
[46:54] We use some of the native stuff, but we're basically Unix application ported on top of Windows.
[47:00] Yeah. Yeah. Yeah. Yeah. That's why so I yeah. I saw that quite. You have Tom Ketterner.
[47:06] Yeah. Yeah. Yeah. Yep.
[47:08] That's part of the reason why we cannot use a lot of the Windows services underlying.
[47:13] But that even if you're on Linux, my answer would be the same. We don't support OS level
[47:20] clustering.
[47:21] Our clusters are application level. But it's always preferred to do it on the application level. That's because we have to only
[47:29] recommended.
[47:30] Yeah. The only tested,
[47:32] the only way it works properly.
[47:34] Okay.
[47:35] Well, thank you for the clear answer.
[47:37] Because I was discussing this with a colleague, and they said, why not use a Windows cluster? But
[47:42] I don't think we really we don't really need it, I think. So it was to see if we could do it. But
[47:49] No. Don't do that. Go either active active or enterprise cluster with an external database.
[47:55] I would strongly recommend the the enterprise cluster
[47:58] simply because of how it works. But if you have a small enough environment, and if you can
[48:05] take the
[48:06] manual synchronizations and all that funny stuff, active active is also an option.
[48:12] But I prefer enterprise if you can,
[48:15] not just not because of the pricing or anything. It just is a better model.
[48:19] I know. I think the the third question is, in fact, you can use multiple databases.
[48:25] We would like, for example, to use SQL Server for Microsoft, but do we need an extra license for that? I think you need the enterprise cluster Yes. Of the yeah? So Yeah. If you look at that, then you have to use internal,
[48:38] I think, Maria database.
[48:41] That's correct?
[48:42] So if you do standard cluster, it will the servers will come with their own MariaDB,
[48:48] and you cannot use a different database.
[48:50] And this database can also be changed by X-ray at any time.
[48:55] The same way we change from MySQL to Maria, we might decide to change to something else. Think of that as embedded database.
[49:01] You don't care what it is. It's just there.
[49:04] Okay. Yep. If you want to use an external database,
[49:08] MS SQL, Oracle, Postgre,
[49:10] you need to go for enterprise cost.
[49:14] Okay. So, yeah, that's quite an expensive option, I think.
[49:18] But is there a disadvantage
[49:20] of using the embedded database?
[49:22] Or
[49:24] Yes and no. So
[49:26] if you lose so the biggest
[49:30] there are two big things which are diff which are the main differences. One of them is that in if a server dies,
[49:37] you know, something happens.
[49:39] In enterprise cluster, when it comes back, it will self join to the cluster, say, hello, start working again.
[49:45] While in the standard cluster, you literally need them someone manually or with the script to reintroduce
[49:52] it back into the cluster.
[49:54] Okay. It's not automatic.
[49:56] And the other big difference is the event distribution.
[49:59] In an enterprise cluster, each node is picking its own jobs from the database
[50:04] based
[50:05] on
[50:07] its own load. So if one of the nodes know, oh, I'm busy with a lot of PGPs at the moment, it will not pull new jobs.
[50:14] In the standard cluster, the primary close the primary node is a dispatcher.
[50:19] Mhmm. So it will send jobs to the secondary without caring what the secondary is doing based just on numbers.
[50:26] So if it sends a lot of scary jobs
[50:28] to the secondary, the secondary cannot tell it, oh, hold on a second time. I'm busy.
[50:33] Yeah. So that's the other thing is
[50:37] if you're going to use the API
[50:40] or
[50:41] stuff like that with standard cluster, you need to go against the primary.
[50:46] Mhmm.
[50:47] And you'll need to make sure who is the primary at the moment. And also tracking table and server logs in the enterprise
[50:53] data bay in the enterprise cluster,
[50:56] you can see them from any of this admin UIs
[50:59] because they're consolidated.
[51:01] Yeah. In the
[51:03] standard cluster, every so the server logs will be on whatever
[51:08] server is running the kink, so they can be split. But
[51:12] the packing table stays only on the primary.
[51:16] Okay. Yeah. But I don't think we will go to clustering.
[51:20] It will be a single server setup with an an embedded deep database.
[51:25] Just a small question. If you choose an embedded database, normally,
[51:29] don't need to do any DBA tasks on it. You have a problem, we contact X-ray. That's correct? Or Correct. It's embedded. You are not supposed to touch it. Okay. Yeah. That's clear. Okay.
[51:40] Yeah. Well, thank you very much for all the clear answers.
[51:44] Yeah. Okay. Thank you. Okay. Okay. Kevin.
[51:49] Hey. Good morning. Okay.
[51:51] Fine. Got
[51:53] got two environments, qual and prod. They're all running twenty sixteen boxes. I got four engines on each the the edge and the back end.
[52:02] The
[52:03] edges are running 16 gig of memory, and the back ends are running 24 gig.
[52:08] The problem I've had since I patched the qual with September and then did the October,
[52:12] the edges, when I do the start all command, not all services come up. The DB monitor will and the other ones start coming up, then they drop. Then I had to mount and start them. Once they're all up, but I do the sync all, it works.
[52:25] When I do the bounce all, it fails yeah. Go ahead.
[52:28] Sorry. Are you on Windows or Linux? Windows. Windows.
[52:32] Yeah. I know what the problem is. Well, it's an old problem.
[52:36] It is? Okay. Yeah. Yeah. Because I try to I try to update monthly. I try to keep the updates current. Yeah. So it has nothing to do with the update. It's basically timing on your box itself, and it sometimes will happen, sometimes will not. So the way it works, it's supposed to work, is that the database need to start completely because any of the before any of the other services try to start. Mhmm. If any of the others try to start while the database is still stacking,
[53:01] they will fail, and you need to start them manually. Ah, okay.
[53:05] And that's what get going on on Windows is that because the new build is actually a
[53:11] little faster than the old build,
[53:13] they come up a lot faster. What I would I would usually recommend Windows customers to do is to set all of our services to manual
[53:23] Right. And create a new service
[53:26] which has the database starting
[53:28] followed by a sleep,
[53:31] you know, about thirty seconds or usually less, but, you know, be on the safe side, and then start of the rest of the guys. And that way, you know your database is fully started before anyone else comes up. Okay. Good. Yeah. That kinda seems to make sense what I see observe what I observed there. The back ends are fine. Back ends never gave me an issue. They're fine. Now. Yeah. But they are now. And it is because the database is starting because you have more memory there, because you have a better CPU, because how the networking layer is. By the way, 16 gigabytes, I don't like that. Bump it up next time. Just saying. Yeah. I think I'm bumping up the 24 for the edges. Yeah. Yeah. And a little higher also down on the edge on the servers. But the point is that might start happening on any Windows server at any time. It really is about timing.
[54:18] Just how fast the database starts.
[54:21] Sooner or later, you'll hit that problem almost anywhere on a Windows environment.
[54:25] And the more updated you get, the more likely for it just to happen because we're a little faster in getting to the next process because Right. You know, we we are
[54:36] we're trying to start as fast as possible. And as soon as the database process so the way it works because they're out if they're automatic, it's even worse because you never know what starts first.
[54:47] If you do them basically,
[54:50] make sure that
[54:52] it's a start all
[54:54] script.
[54:55] The other thing is
[54:58] on usually, just bring them in order as soon as the previous one says, I'm done. The database is usually up, but on Windows,
[55:05] specifically, I've been seeing that a little
[55:09] slower.
[55:10] So start all always works. Right? So that's the other thing. Instead of ordering them with the stop, and so I won't do the start all. Instead of letting gauge service on its own. So build a service that just calls the start all.
[55:23] Okay. Because start all orders them properly. Something like that. So that's an old problem on Windows because unlike Unix,
[55:32] where you can order processes,
[55:33] you when you do Windows processes, you know, They basically are trying to start all at the same time. And if your database is not up,
[55:41] the other thing, if it's only on the edges, check how big your database are Okay. They had become. It's possible that because they had become a little bigger,
[55:52] they they're starting little slower.
[55:54] See where I'm going with that? Also,
[55:57] you probably had applied
[55:59] some database to make connections and so on down on the servers, but not on the images because there is no transfers up there. Right? Right. Look at those, though, because you might want to update your database your database configuration up on the edges as well
[56:15] so that the startup doesn't take that long. You know? There is there is caches and memories over there. Maybe bump those a little bit to get the database more power on startup,
[56:26] so it starts a little faster.
[56:28] But still,
[56:29] you need to make sure it started completely before anyone else tried to start.
[56:33] Right. Yeah. It's funny, though, the edge, when you do the sync all, it works.
[56:38] And only edges, when you do the bounce all, it fails on the secondary, so I had to manually bounce them. Oh, that's normal because there's no TM. Just ignore that part.
[56:47] It's not a cluster on the edges. Right? It's just a a synchronization
[56:51] mechanism.
[56:52] Mhmm. So the bounce can be a little weird up there. That that's a kinda known kinda. Sometimes it works, sometime it doesn't. I I wouldn't read too much into it. Yeah. No. I don't. No. I don't. Now the other thing is I patched my production with the December patch.
[57:07] And to do that because of the update to the MariaDB,
[57:10] I had to drop the AV. Yep. And it got where it got it done. We patched it. Then all of sudden, we had problems, not everything, but several
[57:18] folder monitors all of several folder paths. I couldn't write to it. I got access denied.
[57:23] Not all. Just, I would say maybe about five paths,
[57:27] and it drove us crazy. And we end up back in. We backed the patch out. We went back to October,
[57:33] and it resolved.
[57:34] So I'm hoping that the February patch will resolve this.
[57:38] Yeah. One thing I've seen on a couple of Windows updates lately,
[57:42] the prob do you have anything
[57:45] like Windows Defender running or something like that? We have carbon black as far as the AV. I they might I have to check if Defender's running on it. Yeah. Make sure it's fully disabled. And the other thing that we noticed,
[57:58] make sure you're running your updates with the local admin and not with a l dApp or domain admin.
[58:06] Right. Okay. Which I do. Yeah.
[58:08] Just make sure someone didn't screw up that. Because the last time, they were telling I had a Windows update last week that was not going well. And they kept telling me, yeah, it's a local admin. Then, no. It's the domain one, but we treat it less a local.
[58:23] Yeah.
[58:24] No.
[58:25] I really need the local. Yeah. And I did the January patch on my call system. Mhmm. Everything came up fine. The only thing is all my FTPS connections were failing through a socks malfunction.
[58:37] So I I had to back that out. So I'm hopefully, I'm saying February does resolve this. Because years of month months ago, there was a problem with FTPS
[58:45] on one of the patches, and they said they had a a flaw in it. Wait till the next patch comes out. And it came out. I got it worked.
[58:52] Yeah. It it happens occasionally, but also checks the security.
[58:56] FTPS
[58:58] sometimes fails when we bump a little bit the
[59:03] ciphers.
[59:05] So
[59:06] Okay. Alright. Excellent. Thank you very much. Good.
[59:09] Okay. Joe Joe Campos.
[59:11] Hi.
[59:13] Actually posted in the chat, but,
[59:16] basically,
[59:17] we are always looking for opportunities to enhance our,
[59:20] availability
[59:21] of the MT infrastructure.
[59:24] And we're running, enterprise cluster
[59:27] in an active, passive
[59:30] dual data center environment today,
[59:32] and we do a synchronization
[59:34] for user accounts. They're independent,
[59:37] environments, and we do synchronization of the accounts, between the two environments for our Doctor purposes.
[59:42] But I am looking at the potential of using
[59:46] something like the zero downtime
[59:48] active passive within data center to
[59:51] enhance that.
[59:53] And I just wanted to understand,
[59:55] you know, what has been the feedback so far from customers that have started playing around with it.
[60:02] None that I had work yet. Let's say it like that. Give it some time, Joe. We're not there yet.
[60:08] Do we have a a feel for when that's gonna be GA? I know it's currently in beta.
[60:15] It depends on how the beta goes. It depends on what is discovered,
[60:19] and it depends on how it goes is the answer.
[60:22] If you're playing with it, if you're starting to look into it, please, please, please, please communicate
[60:27] to us.
[60:28] K. Will do. So look at it whatever that you don't like, whatever it feels like it not doing what you think it should be doing, just talk to us. It might not be feasible,
[60:38] but it any
[60:40] part of the reason we have the betas now coming with every update is exactly to give an opportunity of people to start looking into that. Alright?
[60:48] So that's the time when you can mention what is working and not working for you.
[60:55] Alright. Very good. Thank you so much.
[60:57] Okay.
[60:59] And, Jeremy.
[61:02] Hello.
[61:03] Hello. So I have a I
[61:05] have a question about the basic protocols
[61:09] and
[61:10] how to handle multiple connection.
[61:14] In fact, we have one partner that send us
[61:18] multiple files at the same time. I I would say around 50,
[61:23] which are really big, like,
[61:25] two two gigabytes.
[61:28] And
[61:29] we would to another partner
[61:31] that only supports
[61:33] eight simultaneous
[61:34] connection.
[61:36] And,
[61:38] and we
[61:40] I I think ST cannot update this kind of
[61:44] of routine or I I don't know how because
[61:48] there are eight files that go to the partner, and all the other are failing
[61:54] around the the time I have I have set up. I I don't remember. But after two minutes, the
[62:01] the
[62:02] the the files are are are
[62:05] cancelling.
[62:06] And so I'm wondering what is the parameters that I have to to set up to to go on a
[62:11] on a full
[62:14] to to to to put the files on the on the queue.
[62:18] Okay. Can you see my server?
[62:20] Yep.
[62:21] Okay. So okay. Let's ignore that for now.
[62:26] Let me
[62:27] there is a couple of ways
[62:31] to do that. So
[62:36] come on.
[62:41] So when you create the transfer site,
[62:44] that's the first thing.
[62:47] They don't have one, so I'll just create. So it's valid for any protocol, not just for the PECID. So this will be different from what you're used to with PECID. It's not like the CFT PECID connection
[62:57] because
[62:58] we don't have the same type of queue. However,
[63:01] if you look look at the top of the site,
[63:05] see where it says maximum? So this one
[63:09] whoops.
[63:15] For Posit.
[63:17] Posit.
[63:17] Posit. Posit. Posit. Hold on. I'm doing something stupid here.
[63:22] Of course, I am.
[63:25] Hold on a second.
[63:29] You say you don't expect to see the questions.
[63:50] For any other prod call, it's on the pro you can do it both on the transfer site itself, and then you can do it on the
[63:58] subscriptions
[63:59] and the routes. But for Peset,
[64:04] is that are so you are worried about
[64:07] the inbounds or outbounds?
[64:10] No. Advance.
[64:12] Because the the inbounds are are
[64:14] I think, infinite.
[64:16] We have simultaneous
[64:18] simultaneous
[64:19] transfer to to zero, and it it seems to work. Okay.
[64:30] So this is for the inbounds.
[64:33] And this one, you say you don't have a trouble with. Right?
[64:37] Yep.
[64:38] Outbounds?
[64:40] See, I don't remember. So let's try to figure out where it was.
[64:44] I I think it it in the in the passage to transfer profile.
[64:48] You That's what I think it is.
[64:51] Yeah. You you have to to check the two advanced settings, and you would see the the parameters. But I think it's not working. In in fact, in my configuration,
[65:01] it didn't.
[65:02] So what what
[65:04] update are you on?
[65:08] I'm on the
[65:10] I think,
[65:11] April,
[65:14] 02/07/2022.
[65:17] K.
[65:23] Okay. My server is misbehaving,
[65:25] obviously.
[65:30] You know what? I don't know off the top of my head.
[65:34] I will take a note, and I'll try to get to you later today. Okay? I just Okay. Don't know. So technically speaking, I know that we have a setting somewhere,
[65:45] and we can throttle it. And if it doesn't work, check with support if you don't need to update because we had been having troubles with some of those specific settings.
[65:55] Okay. But you you you said something
[65:58] is,
[65:59] is working,
[66:00] about this, this queue or this plotting
[66:05] Yeah. So the way Okay. Yeah. So the the way SQ works is that if there is too many files to go to the same place, they automatically
[66:15] go to what we call the event queue, which is our internal queue. It's not like the CFT queue where you can see things and you queue things.
[66:23] In SD, you just send them out, but then they go as events in the database, and then the database says, oh, I have availability.
[66:30] Now one thing you can try
[66:33] in the meantime is if you go to your route
[66:38] on the send to part so are you sending with send to partner, or are you sending in a different way?
[66:44] How are you sending all the person? I'm I'm sending to to partner with
[66:50] with the specific
[66:51] steps here. Yeah.
[66:53] Step. So if you look at the send to partner,
[67:01] No.
[67:09] Those will not help you.
[67:12] Okay.
[67:12] Let let me look into that. There was I think you're right. It's our profile, but it's not loading for me for some reason.
[67:19] If it's not working, check with support. But
[67:22] so when the file arrives, we'll process it and we'll put it into what we call our event queue.
[67:27] After that, the servers will start pulling from there. How many they pull per side depends on the configuration.
[67:33] For the other protocols, the configuration is on the pro on the site itself that says how many is the maximum number of files moving towards that in that direction at any time. That's our throttling.
[67:47] For for PEC, it's somewhere on the profile. If you had set it up and it doesn't work, with support.
[67:54] Okay.
[67:55] And I know this didn't help much,
[67:58] but I
[68:02] don't I haven't used that one in a while. I know it's there because I know I've restricted it before.
[68:09] Hey, Annie. This is Jean.
[68:12] I think it's network settings.
[68:13] Hi. I think it's network settings on the Pesset transfer
[68:17] site.
[68:19] Oh. All the way Okay. All the way at the bottom. Isn't it simultaneous transfers?
[68:23] That's the one. Thank you, Jean. You're welcome.
[68:27] I'm paying attention.
[68:30] It's always good to hear people paying attention
[68:32] even when I'm not.
[68:36] See?
[68:37] I don't remember everything.
[68:47] Yeah. You you have to check
[68:49] advanced transfer. Show advanced transfer.
[68:52] Here we are.
[69:01] Here they are. So
[69:03] did you set this one?
[69:06] Yeah. This is the one, and this is not working.
[69:10] Okay.
[69:11] Support case. It should be.
[69:13] Okay.
[69:14] So if it's not working, then something they broke something something something will happen. But the idea thanks, Jean, again, for reminding me why it was hiding.
[69:23] But
[69:24] this is what will
[69:27] should be restricting it.
[69:29] And if it's not working,
[69:31] then the mechanism internally
[69:33] now
[69:34] the one thing you're in a cluster. Right?
[69:39] Yep.
[69:41] Drop it to how the value you want and see if that will work. I'm not sure if it's counting for both servers or just one of them.
[69:53] So what happens if you set it to, for example, just two? So take it just to two and see what happens because it's possible that it's working, but it's not counting. The other thing is with all of those values, I strongly recommend people to leave 10% of grace.
[70:09] So if you want 10 transfers maximum,
[70:11] don't put more than eight in the value over there just for
[70:16] counting purposes because, you know, we you might not be closing properly at the end and so on. So drop it a little lower and see if it that will fix it.
[70:25] And if it doesn't check with support and see what they'll tell you. But this should prevent
[70:31] this transfer site from being used for more than that number of transfers at the same time.
[70:39] Okay.
[70:40] Thank you.
[70:41] Yeah. That that's the best I can tell you. And, of course, bugs happen all over the place as usual.
[70:49] Other from that and I know that you can do that, and I know this used to work because
[70:55] one of my test in my lab, I have a small CFT. You know, one of those three trans only three connections allowed.
[71:02] I know that I've sent thousands of files into it for testing. This is when I was testing Pacific
[71:08] 3 at a time. So
[71:11] you might need to update.
[71:13] Okay.
[71:14] With this being said, yay, no one else gets an hands up. So I'm going to the chat, guys, and we'll see if something else is there. And sorry about, you know,
[71:25] many questions. I haven't got to the chat yet.
[71:30] Byron, that's a question that's about row row access and cloud that we covered, I think.
[71:36] Anthony,
[71:38] thanks
[71:39] for saying for mentioning about your s three.
[71:43] Joe, did I answer all your all your questions?
[71:49] Joe Campos?
[71:52] Think he's good.
[71:57] Thomas, we talked about updates.
[72:00] So all set on that one. Right?
[72:04] Okay. Alexis,
[72:06] opportunity on space usage.
[72:08] As all the patches are stored locally on the system and consuming space, that will allow to remove the old patches that have been applied. Thomas, yes.
[72:17] Oh, sorry. Not Thomas. Alexis.
[72:19] Yes.
[72:21] However,
[72:22] check,
[72:23] make make sure you never want to back out back into them.
[72:27] Basically, if they are not there and you need to back what I would recommend actually is to just zip them and move them away,
[72:35] but restore them before you do the next update just in case we check for something.
[72:41] Make sense?
[72:46] Alexis?
[72:50] No one is talking to me.
[72:53] Okay.
[72:57] Mark is talking about answering the question about the file maintenance we talked about
[73:02] with the scaling.
[73:05] K.
[73:12] Next.
[73:15] They
[73:15] want question.
[73:17] How to get email notifications enabled for list of everything that was deleted?
[73:23] They, you still on the phone?
[73:32] So the question is how to get email notifications enabled for list of everything that was deleted. Unfortunately,
[73:38] you cannot do anything like that because we don't send
[73:42] notifications
[73:44] like
[73:45] that.
[73:49] You can get notifications
[73:51] when something is about to expire, but not on the depletion itself.
[73:57] The
[73:58] easiest way might shot
[74:00] the snapshot of what is after.
[74:02] Hey. That's you.
[74:05] Oh.
[74:11] Or also the audit log. The audit log might be your best friend for that.
[74:21] Charles,
[74:23] testing.
[74:26] Did you get any answers on that one?
[74:29] The automation,
[74:31] simple test.
[74:33] Sorry, guys. I'm just trying to go through the
[74:39] logs.
[74:40] And
[74:41] Mark, Bill, thanks for sharing the screen of how to get some notifications,
[74:46] the ones that can be set up, actually.
[74:53] Ray, you still around?
[74:57] Are you asking about deletion from the file maintenance,
[75:00] or are you asking about deletion on anything on the server?
[75:19] Anyone, can someone say something? I'm feeling glad I'm talking to myself at the moment. I can I can hear you? I can hear you. Okay.
[75:28] I was I I think we covered most of what is in the chat on the phone as well.
[75:34] So but I'm just looking at okay. Charles,
[75:37] email messages from the admin service indicating the server log database is unavailable, and then a few minutes later, back to normal.
[75:45] These notifications
[75:47] oh, Charles,
[75:49] Mac, l I cannot pronounce your file last name.
[75:54] Are you on enterprise cluster or standard cluster?
[76:06] I believe they have left.
[76:09] Yep. It looks like it. Think it is the enterprise cluster.
[76:14] Yeah. So it
[76:16] so if it is enterprise cluster and the database is reported occasionally as missing
[76:21] people, just look at your Guinness and networking settings
[76:25] as he cannot find something.
[76:27] For standard cluster, if it is on the local and the database cannot be found, it usually means an overloaded machine.
[76:34] Like, it bay basically, the database is there, but the connection is slow.
[76:44] But
[76:44] Yep. It is this recommended recommended to install Xway other Xway products on the same machine.
[76:51] Please don't do that.
[76:53] Okay. So treat ST as a
[76:57] very
[76:59] let me see how to say it politely. A very greedy sibling that really doesn't play well with anyone.
[77:06] Mhmm. Yeah. So
[77:07] it's a very resource heavy server. So anytime when you install something else on the same machine, it will eat from the same
[77:16] resource pool. Right?
[77:18] So when that happens, you never know what performance will be there. And even if we're talking about something as small as a CFT, for example, our CFT,
[77:27] or Connect Direct from IBM, you know, one of those guys. Even for them, I strongly recommend them to go on their own servers and to connect to them via a load balancer for high availability.
[77:37] Okay. The only things that need to be on the same server at ST, one of them is our event router. If you're using Sentinel or MFTOI
[77:46] or, you know, one of our visibility packages,
[77:49] installing an event router on the same box that says, it actually is helpful
[77:54] because it eliminates the the networking hop that ST needs to do to go to the visibility. So it reports locally, and then the event router is the one routing out.
[78:05] Right? So this is allowed. And any monitoring tools that don't disturb ST itself.
[78:11] You know, always level monitoring.
[78:13] So Well, we are using X-ray Sentinel,
[78:15] so that's not an issue. Yeah. So x Sentinel needs to be on a separate box? Yeah. Event event router
[78:24] that talks
[78:25] that usually you can, and I strongly recommend to put between ST and Sentinel
[78:30] will need to be on the ST box.
[78:33] That's the thing you need to put there.
[78:35] Because that way, your connection between the event router and ST becomes local, so less prone to travels.
[78:42] You keep the big queue the big
[78:45] circular queue actually into the event router. So the ST one is a very small one.
[78:52] And if you ever need to you if you know your signal will be down for six hours,
[78:57] you can increase the queue
[79:00] in the the the buffer in the event router. We just had a start of the event router as opposed to needing to restart SD itself.
[79:08] Okay. Yeah. So that's what you can put on this box. But any other transfer software, stay away.
[79:16] Anything else that is not
[79:18] security. So if security wants to put a monitor for networking, you know, something like that, of course, you can do that. You know? It's 2023
[79:28] already.
[79:29] Yeah. I was going to say 2022.
[79:33] But it it's it's but nothing
[79:36] heavy. If you need to put a client for something,
[79:39] you know, do that. But Yeah. No other listeners,
[79:43] nothing that can eat out of the resources. And part of it is because
[79:48] no matter how you do it, the resources will be shared.
[79:51] And chances are they'll hit at exactly the same time when you need ST to be at high performance.
[79:56] So just stay out.
[79:59] Yeah.
[80:00] Yeah. I thought so because we tried to install it on the test server for a book. And then Uh-huh. Problems with the class part and multiple GDKs installed. And, yeah, it was It becomes a nightmare.
[80:13] Yeah.
[80:14] Yeah. It it yeah. And Olga, yes, that's exactly what I am saying. The only two products from Maxwell that can live on the same box is one of the hours servers and event router
[80:25] that allows you to go to send email and so on.
[80:29] Anything else, two of them should never ever live on the same box. And for antivirus software, we also had a problem during the installation
[80:37] with
[80:39] it's also called Sentinel,
[80:42] another product. And, yeah, we had quite some problems with it. We had to deactivate it
[80:47] during the Oh, yeah.
[80:49] Are there any practices,
[80:51] what we need to do to
[80:53] to
[80:54] to have those two products together that there's no conflict between
[80:59] You cannot
[81:01] Because so so the way ST works is that we touch a good a lot of files very a lot of the the files a lot of times. During a normal upload, for example, we will touch the same file about 20 times.
[81:15] It's not just an open channel. We keep checking. Is it okay? Is it that? Is it that?
[81:22] During installation,
[81:23] what is happening when antivirus
[81:26] or Windows Defender or anything else is enabled
[81:29] Mhmm. Is that
[81:32] they
[81:34] lock the file for a millisecond just when we need it.
[81:38] And because we're deploying a gazillion of very small files, sooner or later, it fails. So the basic rule for ST deployments is
[81:47] all the antivirus and security softwares need to be turned off completely
[81:52] during installations and updates.
[81:55] Okay.
[81:56] And during the operation of ST itself,
[81:59] all the directories that ST holds, which is usually, you know, your install directory and your storage directory,
[82:07] should be ex excluded
[82:10] from any antivirus,
[82:12] DLP,
[82:13] whatever is on the ground.
[82:15] Basically, don't let Windows lock a file on us because the moment when
[82:21] and this is especially valid on Windows because of how Windows locks work.
[82:26] On Unix, you can have two processes touching the same file without a trouble. Right?
[82:31] If you want to do a security scan of the incoming or the outgoing files,
[82:37] is this possible to have some integration between
[82:40] s t and some antivirus products? Yes. But yeah. Yeah. This is iCap.
[82:45] So if you look into our
[82:48] admin guide around admin UI and I have some but why I don't you see my server. Right? Yeah. Yeah. Yeah.
[82:55] So if you go to configuration,
[82:58] see where it's we say it's iCAP settings?
[83:01] This is where you specify your iCAP your, basically,
[83:05] antivirus server.
[83:07] And then you can apply a policy to say
[83:10] which of which files to be
[83:13] scanned and when to be scanned, and that will be happens. So the way it works is the file arrives, then we send it send the part of it that is required over iCap
[83:22] for scanning, and then we don't process until the scanning
[83:26] clears.
[83:27] And this iCap is supported by most antivirus
[83:30] products or, for example, ClemV,
[83:33] ClemAV. I I use it in the past, but it's a Unix product.
[83:37] If you use Sentinel,
[83:39] I don't know if you know the product.
[83:42] Their support for that with iCap. Let's
[83:46] say it like that. If the antivirus
[83:48] server isn't it is from the last 10 and doesn't support iCap, someone over there is not doing their homework.
[83:55] Okay. So it should be This is yeah. ICap is the standard
[84:00] protocol
[84:02] for inspection. It's both for DLP and antivirus, and we can support it both.
[84:07] And when you enable in ST, you can enable on inbound, outbound. You can exclude big files. You can exclude specific partners.
[84:15] So it's very configurable, and you can almost configure it down to per user if you want.
[84:20] But it means that you are sending the file to an antivirus server,
[84:25] and it's not an
[84:27] agent that is running locally on the server?
[84:30] Correct. It's not local.
[84:32] It's always
[84:34] outside, and you connect to it over iCap.
[84:37] Okay. And the performance impact is also quite if you have a lot of big files, then you need and yes
[84:43] and no. It's actually smaller files are your bigger problem. You don't send the complete file. What you send is something that is basically the review size, and you specify the
[84:53] server tells you. So when you configure an AICAP server, it will tell you it will ask you what is your review size,
[85:00] and, that will come from your,
[85:02] AI antivirus people. They'll tell you what is their review size. It depends on the server. Usually, it sends about two megabytes
[85:09] at the most.
[85:11] Okay. Viruses are not hiding down in the files. They're up on the headers. Right? So I'm seeing, Yeah. That they identify the file and Yeah. Which I look at it up in a database of known viruses. Yeah. Yeah. Yeah. Yes.
[85:24] One thing to be very careful about because of where it's sitting, it's basically going
[85:29] into antivirus before
[85:31] any of the processing starts,
[85:33] which means that if the file arises a PGP file, we will send to the ICAP or the PGP file and not the unencrypted file, which it cannot touch.
[85:42] Okay. So if you use PGP and antivirus,
[85:45] you will need to do a middle loop to re transfer the file after you decrypt it so that you that can actually be scanned if you want to scan it.
[85:54] Okay. Yeah. So it's not straightforward.
[85:57] But and that's what so on and that's what I started saying. On Linux,
[86:03] we don't like antiviruses
[86:05] touching the files on the storage either, but they don't lock them. So, you know, good enough sometimes.
[86:11] On Windows,
[86:13] the files get locked.
[86:14] And the way we work is that even though we say that's the file it arrived, we have about 20 different things to do on that file later. Mhmm. Yeah. And each of them says, can I still touch the file?
[86:27] And if antivirus has it, we cannot. It will fail.
[86:31] It's very hard to troubleshoot.
[86:33] So the basic rule of ST deployments is keep your security and
[86:38] whatever software, including antivirus,
[86:41] away from our files. And if you want to scan something, do it via the iCap connector.
[86:46] Anything else
[86:48] is a recipe for disaster.
[86:50] And the bad news is it might work in development.
[86:54] We've had people that tell me, well, it works. Sure. It works until it doesn't.
[86:59] Because sooner or later, it's timing.
[87:01] Yeah.
[87:02] So Yeah. I mean, during my POC, we reactivated the antivirus, and we don't see any problems. But, yeah, I need to review with
[87:11] our antivirus guys what we need to do and look at the eye cap settings.
[87:16] So, yeah, it's very clear. Thank you for the good advice.
[87:21] Yep.
[87:23] Okay.
[87:24] More
[87:25] questions. And I know we're overtime,
[87:27] but we have questions. So I'll just finish that.
[87:32] And
[87:33] that always happens with this course anyway.
[87:36] Okay. Where was I with the questions? Hold on.
[87:39] Maybe I was struck somewhere.
[87:44] Thanks again.
[87:45] Noah posted one of the ideas they have.
[87:50] No? The idea that the ideas Porto was talking about, and Nicole will talk about it a little bit more a bit later.
[87:56] Errors after changing the default admin credentials after applying the December 2020 patch.
[88:04] Clayton, where are you seeing those errors and when exactly?
[88:12] Seeing those within our
[88:14] server logs, also in the file tracking logs, but
[88:18] it happened in our test and our prod environments.
[88:23] I think try. Oh, sorry. Go ahead. What
[88:27] cluster do you have?
[88:29] Standard.
[88:32] You changed the the
[88:34] admin credentials?
[88:36] The admin admin credentials?
[88:39] So, yeah, one of the, like, features, I think, of the December
[88:43] patch was
[88:45] informing us if there was default credentials still in place for those admin default admin IDs.
[88:53] So I was following one of the x-ray's articles to change those credentials.
[88:58] And after changing those credentials, I started getting a bunch of errors just pretty much doing basic FTPS
[89:05] file transfers.
[89:07] Mhmm.
[89:08] So I eventually reverted the change because I I was working with support and we couldn't really figure it out.
[89:15] And then the second try in the test environment,
[89:18] it actually worked and I haven't had issues in the test environment since.
[89:21] But then I moved forward with the production environment and the same thing happened where I was getting the errors.
[89:29] Check which admin account you're using for the cluster communication.
[89:36] Okay.
[89:37] I would strongly recommend to create a separate admin called cluster or whatever, you know, something that is not called that. It's usually using the admin account itself, admin admin.
[89:48] So
[89:49] create a new admin called cluster or services, you know, whatever you want to call it, and change your
[89:57] the the account you used.
[90:00] And let me see.
[90:02] So it's it's in the server configuration
[90:05] itself, not not on the clustering page.
[90:08] And try
[90:09] to do it with a different admin because I've seen that happening when the admin account is not fully synced.
[90:15] So one of the cluster notes
[90:17] basically,
[90:18] the the cluster is a little confused of what the credentials are to talk to its partner.
[90:24] Okay. Cool. That's that's the only place where an admin credentials come into play during file transfers. And that's why I'm bringing it up. Because otherwise,
[90:33] the admins themselves don't play at all on file transfers.
[90:38] Okay.
[90:40] Okay.
[90:42] Awesome. Sounds good. Thank you. Charles,
[90:45] we have recently started noticing the TM rules for sending emails or sending multiple email notifications for both successful and failed.
[90:53] I haven't heard of that, but it's possible that we messed up the values. You know, how there are two values over there to check
[91:00] if the files are,
[91:02] if it is a permanent failure or not, if that's the problem. The other option is that for some reason, your server is actually retrying an event that it shouldn't.
[91:13] So Yeah. The the the re the rule packages are
[91:18] the syntax is they're older rules and and and syntax has worked in the past. So Mhmm. We're we're seeing we're seeing something that used to be a certain way and not and no longer working that way, and that only came as a result of
[91:31] noticing that email notifications
[91:33] with advanced routes
[91:35] are not based on
[91:37] one to one relationships with file transfers. They're based on did the route trigger successfully or not.
[91:43] And the support people want
[91:46] to use the email notifications at this point for their audit records. So they have to be, you know, 20 files went in, 10 transferred successfully, three failed, you know, whatever it is. So now they're getting on a successful transfer, they're getting 12 or 13 notifications.
[92:02] And it's sort of just blowing everybody's mind that this is going on, and the actual engineer that I'm working with has been able to recreate it, but essentially came back at this point and told me,
[92:13] TM rules are being deprecated. Don't use it. Go back to using advanced route. And advanced route doesn't give you what you need, and, course, it doesn't cover basic subscriptions at all. So
[92:24] it's a little
[92:27] less than useful, I should say, at this point to
[92:31] Yeah. That type of of a circumstance.
[92:33] I haven't heard anything about this happening,
[92:37] but the way it's working is that it basically triggers, you know, specific time in the chain.
[92:44] If it's a learning more than once so,
[92:47] if you look at the log files in the lock you're seeing, it's sending multiple times. Is it from the same thread or different threads?
[92:54] Have you noticed? Well, when you yeah. When you go in and and look at the, the server logs, you can,
[93:00] well, I don't know about a thread. You can put in the session ID and you get the whole sequence. Right? So I've created specific test accounts and dedicated rules to
[93:08] so I'm not bombing production support personnel.
[93:11] And I've been running the test and they they all come out so quickly. And and then even what happens then is that the content of the emails
[93:18] within the one transaction
[93:20] are inconsistent.
[93:22] The first one will contain a file name, and it'll contain a transfer site. And the second one will not. It'll have environment variables in that place instead,
[93:30] or it'll be blank.
[93:32] Same email template.
[93:33] Same transcript. But it's Okay. Go ahead.
[93:37] So
[93:38] I I know that we are way over time. Can you do me a favor? Send me your TM rule. I want to look into something.
[93:45] So the way you're describing
[93:47] it sounds like
[93:49] either the TM rule got corrupted.
[93:52] Basically, it either doesn't pick correctly the event type or something work like that. So one thing I would say if you want to troubleshoot on your server,
[94:02] open the you you know the
[94:06] the h x HTML template
[94:09] that shows so you the
[94:11] add to that the environment variable d x agent underscore
[94:16] type and d x agent underscore event. I can get names.
[94:23] So what there are two options here. Either we're triggering on events we shouldn't,
[94:30] or we're triggering multiple times.
[94:33] Because you're telling me that those are different,
[94:36] it starts sounding like we might be triggering on events we shouldn't.
[94:43] So try to print d x agent underscore
[94:47] type
[94:48] inside of the email
[94:49] and see if it comes up as different values on each of those mails.
[94:55] And if this is the case, we'll see if we can figure out what's going on. And in the meantime, I'll see what you are on November. Right?
[95:05] Yes. I believe the production is
[95:08] is that we were told to hold back on the one because of a known bug in December. So we're getting ready to apply
[95:15] January,
[95:17] not this weekend, but next weekend.
[95:20] So I
[95:22] I have a server with the I I don't know which release it is, though,
[95:28] that that I'm using
[95:30] team notifications a lot, and I haven't seen that happening.
[95:33] So as I said,
[95:36] my good feeling is that something got corrupted somewhere, maybe in the team rule itself,
[95:40] maybe in the way we're passing the team rule.
[95:43] But until we figure out if it is on step if it's throwing the wrong key on wrong key events, So if the condition is broken
[95:53] or if we're just running it multiple times,
[95:56] I'm not sure we can figure out something. The fact so you said
[96:00] multiples per for every transfer.
[96:03] That will happen if the event is not caught properly.
[96:06] You know how in the if you look at the condition, it say if the event is this and this and this. Right?
[96:12] I think maybe that first part is getting broken.
[96:16] So that's why you will be seeing with different values every time because each of the events carries only part of the environment. What only what it needs.
[96:25] Makes sense?
[96:27] Yeah. Kinda.
[96:29] So let's try to figure out first mhmm.
[96:32] Yeah. I'll take a look at that and see if but I like the idea of being able to print that out in each instance and at least
[96:38] Yeah. See further Have you ever what's?
[96:40] Yeah. Had you ever played with the dump ENV script?
[96:49] I'm sorry. I didn't get that last question. Or Had you ever played with the dump ENV,
[96:54] dumping environment script?
[96:59] That doesn't sound familiar. Yeah. So we have this small agent
[97:03] out of the box that can be put just before the mail notification
[97:08] that instead of changing the x h t m l, will basically dump the whole environment into a file on the file system.
[97:15] So if you inject that before the mail,
[97:19] it will also create a file for each of the mails it's sending. And then when you open it, it will give you all the variables, so you will be able to see what is inside.
[97:28] If you never played with that, ask support to assist you to put it on your mail notification package.
[97:36] Okay? Okay. And because this will tell you when we are firing the mails and what is in the environment at that point. And knowing what happens will eventually be able to figure out what is happening and why.
[97:50] Because at the moment, it can be almost anything.
[97:54] It can be event retry.
[97:56] It may be a failure. It may be as far in the same event multiple times, which shouldn't be happening because you have other troubles. But it also might be just the erroring of the condition. And if it is the erroring the condition, that might be the easiest to solve.
[98:10] But we need first to catch it. So get dump e m v. It is d u m p as in Peter. It's like dumping the environment. That's what it does.
[98:21] So ask them how to apply it on your mail.
[98:24] If you have never played with that, it will create
[98:27] every time when you send a mail, before that, it will also create a file. It will go into your install directory slash bar slash t m p. They're named dump m v dot the ID of the process, so they're not easy to recognize. But, you know, on the test server, we won't be running thousands of files. They'll need to do a couple just to see. And then look to them and see what they'll tell you or give them support.
[98:49] That's the best I can tell
[98:52] without hands on the server.
[98:55] Very good. Well, that's actually very helpful. Thank you. Yeah. I and that that's basically if anyone is working with the tier notifications notifications as much as we keep telling people not to. We know that for some reasons they still need to be. That's the way to troubleshoot.
[99:08] And that also will help you see what variables are available because then what that's what you can use down in x h t m l. Right?
[99:16] So
[99:17] don't panic with your best friend. And don't forget to turn it off after you
[99:22] finish working because it will be creating files like crazy.
[99:27] Okay.
[99:29] Just think.
[99:30] Okay. I know we're way overtime, Nicole. I'm sorry. We're chatting. Okay. Before you take over
[99:38] You're right. Sorry. Customer's Yeah. Questions,
[99:41] so I cannot blame you, Annie.
[99:44] Before she takes over because okay, guys. Last call. Any last short questions?
[99:51] Any Very short,
[99:53] please. Yeah. Yeah. Yeah. Because I know it's
[99:55] it's frustrating. I have a really short question. This is Lisa from Franchise Tax Board. Hey.
[100:01] Hi.
[100:03] Just quickly, what are your, recommendations
[100:06] for, applying the patches or updates?
[100:11] What do you mean? Monthly,
[100:12] every other month? We
[100:15] had that earlier before you guys joined.
[100:18] But, basically, what I keep telling people,
[100:21] plan for quarterly,
[100:24] at least.
[100:26] This will give you at least two per year
[100:29] if you need to skip some. Don't wait too long. We I r and d will tell you do it every month. We know that will never happen.
[100:39] So plan to do it every two or three months or something like that.
[100:45] Great. Thank you.
[100:47] I know. And I I can get in trouble for that answer.
[100:51] But the reality is the reality is no one is going to update every month. We all know that. You don't have the processing power for it. If you can do it every month, be my guest. You know, hire someone to do just that. I'm good.
[101:03] But real life, tight quarterly.
[101:07] And the other important part, and I'll
[101:09] keep saying that, and I said it earlier, but I'll repeat it.
[101:13] Keep an eye on what is coming in every release. Some of them has major security improvements.
[101:19] If one of those hit, try to put it in as fast as possible, even if it is out of sequence for whatever you are doing.
[101:28] Okay?
[101:29] So
[101:31] that that's the best I can say.
[101:34] But Okay. Especially for people with development server,
[101:38] which should be everyone on the phone,
[101:41] but download updates. When you have few three minutes, download and put it on your development servers and see how it behaves.
[101:48] Even if you're not going to apply that and you're going to skip it, just by seeing what it has will be helpful for when you actually go. Remember where that's fully accumulative.
[102:01] Right?
[102:02] So even if you're going to skip, for example, February,
[102:05] download it. Play with it on your development server.
[102:08] That way when March comes,
[102:10] it might be easier for you
[102:13] to actually see
[102:15] what is new and better mind when to jump.
[102:18] So
[102:19] that's the best I can say.
[102:22] As you know, do it as often as you can that is not disruptive.
[102:28] And please, please, please don't wait for twelve months.
[102:31] Something will go wrong.
[102:36] Great. Does this help, Lisa?
[102:38] Yeah. It does. Thank you. It it's,
[102:41] you know We're normally not. We're normally only, like, three to four patches behind.
[102:47] So we're not Yeah.
[102:49] And keep in mind, unlike old versions where we had specific patches that were security only and you could only apply them when they came in,
[102:59] we don't do that anymore. If there is security breach, it will be in the release.
[103:04] So you'll need to go on it. So the closer you are to it, the less likely is that something will backfire.
[103:11] So
[103:13] okay.
[103:14] With that, Nicole is going to start yelling at me. So I'll just give her back to Michael. Okay. So everyone,
[103:23] thanks for joining.
[103:26] And
[103:28] back to Nicole, she has some
[103:31] administrative
[103:32] things to talk to you about from me. Yeah. Very quickly. That was a nice call.
[103:38] It was very,
[103:40] very interactive. I have the feeling that, our customers enjoyed using Teams. I have the I
[103:49] have the feeling that there were it was much more interactive. So
[103:53] we will continue using using Team
[103:57] Teams in the in the future. Thank you very much, Annie. That was a a great call as usual.
[104:04] You may see I'm sharing my screen.
[104:06] As usual, you will be receiving a
[104:09] survey. Please you will have it tomorrow. I'm not ready for today, but please answer. It's very important. We read all your comments.
[104:19] And,
[104:20] also, what
[104:21] I'm asking
[104:22] you all the time is to fill in with topics that you would like to see in future agenda.
[104:30] So thank you for that. You will have an email also with the link to the article,
[104:37] on the community portal that will embed the recording.
[104:41] And
[104:42] we do have another
[104:45] ask any session, but for the APAC region.
[104:48] So it will be February 22.
[104:50] And there is a an in person
[104:53] Sorry, Nicole.
[104:54] Let me just chime in. Guys, when we say I APAC, that doesn't mean you are not invited.
[105:00] It just means it's in the evening Arizona time. It's basically about twelve it's about ten hours later than this session.
[105:08] So if you have APAC offices or if you use working APAC hours, you're welcome to register for that one as well.
[105:16] So sure. Yep. But we don't we yep. We don't send the invitation. We send the invitation only to contacts that are in APAC. But if you see the record the the registration
[105:29] link
[105:30] to on the community portal under the user group tab,
[105:35] you're,
[105:36] of course, more than welcome to register and join the session.
[105:42] And there is an in person
[105:44] that is planned in Seattle,
[105:47] March 7.
[105:50] We don't have a lot of customers registered
[105:53] so far, so we might cancel it if, we don't get more registration.
[105:59] So if you'd like to join, Annie will be there also.
[106:03] It's
[106:04] it's the same kind of session, but in person, you can interact with your peers.
[106:10] So please,
[106:11] register if this is something that you would like to join.
[106:15] And with that,
[106:17] I just want to oh, sorry.
[106:20] Just want to thank you very much, and,
[106:23] thank you very much for joining. And I hope to have you joining
[106:28] the next session.
[106:31] Bye bye. Have a great Valentine's
[106:34] Day. It is Valentine's Day today.
[106:36] It is, isn't it? See? And I'm doing my favorite thing, talking to my favorite people about my favorite topic.
[106:44] Thanks, everyone, again.
[106:47] If you have any questions in the meantime, we have the community portal as Nicole mentioned.
[106:52] And if not, I'll talk to you on the next meeting. And Nicole will have it scheduled before you know it.
[106:59] So thanks everyone. Have a wonderful day and
[107:03] have a wonderful evening if you're in Europe. You know, it's kinda getting late over there.
[107:08] So for the rest of us, we still have a full day to work before we can go celebrate. So back to work, everyone.
[107:15] Thanks,
[107:16] April. Thank
[107:17] you. Bye,
[107:18] everyone.