Nigel Jones: Hello!
Ljupcho Palashevski: Hello! There!
Ljupcho Palashevski: Hi! Everyone.
Daniel Bruns: all right.
Jürgen Hemelt: Hello!
Nigel Jones: Hello there! Yeah, I can tell you all the time.
Nigel Jones: I'm not sure if Mandy or Dan or David will join today.
Nigel Jones: Unfortunately, I won't be able to spend
Nigel Jones: any time really on this
Nigel Jones: I've joined out of interest. But
Nigel Jones: I'm not going to be able to do a lot at the moment.
Jürgen Hemelt: Okay.
Jürgen Hemelt: only bad news recently.
Nigel Jones: let's say it's just changes.
Jürgen Hemelt: but opportunities.
Ljupcho Palashevski: Maybe it's also maybe good for me to mention what is going on I was not on this call for at least I think 2 times in a row.
Ljupcho Palashevski: And that's
Ljupcho Palashevski: yeah. I was just gonna remind people they were calls recorded just in case you want to. Yes, and that that's fine. Because, yeah, this is just the information for relevant for
Ljupcho Palashevski: all others. basically, we are going through some restructuring the organization, and
Ljupcho Palashevski: therefore it affects a little bit hour. contribution to Nigeria on
Ljupcho Palashevski: or it affects our capacity first on how we contribute, then, still in
Ljupcho Palashevski: in a process of deciding how we are going to continue. So I don't have more concrete details. Just I'm saying
Ljupcho Palashevski: that's why I'm a bit last present on these calls. I hope that will change and I hope it will be you the proper in a proper direction.
Ljupcho Palashevski: I don't know right now. I don't have more concrete information. So
Ljupcho Palashevski: I'll continue as it was before, at least for myself.
Ljupcho Palashevski: I'll try to follow where we are and
Ljupcho Palashevski: see what we can do next. Basically.
Ljupcho Palashevski: I I know I had to kind of a assignment on my end, never really managed to
Ljupcho Palashevski: dive into it, so I would like to hear whether there is something you guys progressed on or
Ljupcho Palashevski: some movement there. otherwise. Yeah, the plan on my side stays to to try to actually bring that standalone server.
Ljupcho Palashevski: So just that from my capacity perspective, it was.
Ljupcho Palashevski: yeah, what I said, because of what I said, I was not able to work on this.
Nigel Jones: Yeah. So what I did so last week, we talked briefly about some code I checked in and merged into repository. So it builds.
Nigel Jones: So the build infrastructure is there?
Nigel Jones: And gosh! I'm trying to remember what the code did. So it built a new server chassis. and it just added a module that was starting to experiment with different health endpoints.
Nigel Jones: It didn't have any fancy logic in and
Nigel Jones: it just allowed you to return. Http 200
Nigel Jones: and something different if the server wasn't started, which was helpful for kubernetes when you could use the standard health check features rather than have to issue a command within the container.
Nigel Jones: So it was. It with the idea of what I was playing with at that time was just to figure out, how would I build something to experiment with?
Nigel Jones: that would allow me to do that. And and that's what the code that checked in was. Obviously there's more thought needed about what it means for a service to be to be ready for request to be sent to it. And so we had a bit of a discussion about that as well. I I have a question, Nigel, in in that regard. I I've seen the change the commit while working on the release.
Ljupcho Palashevski: and that that is the building. I should do this slimmer version of a service chat. If I'm not mistaken with this new classify right next to the.
Ljupcho Palashevski: to the, to the existing one.
Nigel Jones: Did I all this through properly like, yeah, I'm struggling to remember, because my heads full of stuff. I I have to learn. that's not Nigeria. I made a change in base Nigeria
Nigel Jones: to facilitate this work.
david radley: and I just can't remember what it is. So I'm just going back and looking in the commit history. And then I can explain it to you, wasn't it? It was to do with this? The default spring build option called plane or something.
Nigel Jones: Oh, yes, that's right, thank you. The way it yes, so so What it was is when we pack the server chefs. When we create the service chassis artefact. we only put the the uber jar in. So this is the spring beautified jar which includes spring boot, plus our actual omeg application, plus all of its dependencies, so that we can just take that jar and run it. And that's how we normally run to serve a chassis. That's the big fat jar.
Nigel Jones: Now there is a build time option that lets you create the same line J, which has got less in, but still goes through that spring boot process.
Nigel Jones: So what I decided to do for this experiment was create a custom.
Nigel Jones: server chassis
Nigel Jones: using a grade or build. And one of the problems there was I wanted the Omeg application.
Nigel Jones: but I wanted to do the spring boot stuff myself. I did it within the prototype repository.
Nigel Jones: And I was getting problems with clashes and too much content being in there. Basically.
Nigel Jones: So what I wanted was just omeg application, because I want the same application that we normally use. And we don't actually package that in any way. So to get it out, you can obviously take the full jail and start extracting it. But when we did
Nigel Jones: when we do some of our uber jars. What we do is we put in both the simple jar as well as the jail with dependencies.
Nigel Jones: But that wasn't the case for this spring module. So the change I proposed for base Nigeria was to also package the slim jar within the same maven artefact.
Nigel Jones: and you can select that by an alternate classifier called plane
Nigel Jones: which is actually the standard spring boot Plugin behavior. it's just that I had explicitly taken efforts to remove it previously.
Ljupcho Palashevski: Okay,
Nigel Jones: just for me to understand, because I'm I'm missing something here. Why, we would like to keep running the same spring boot application called to Mac, Server, platform. for my, for my, for my experiment. this is no, this is not saying. This is a long term. Okay, time is that under the context stand. Now, yeah, it was trying to think about, how do I create a platform? By that I mean the word platform generic. How do I create a thing that I can play about with? to do some experiments
Nigel Jones: where we're at at the moment is we haven't got the launch of code for a single server in there. So I thought, well, I want to build a custom version of platform. I want to add this job one way you can do that is, you can take the existing John, of course, just put an extra one in its place. And I thought, Oh, we're starting to rebuild this and we're starting to look at changing what the chassis is.
Nigel Jones: Why don't I take the approach of going? Let's use what I can from existing artifacts in the base. And then let's put the extra stuff I'm experimenting with in this new repository. So that was the thought process for this point in time for why? Why I did what I did in there.
Ljupcho Palashevski: Okay.
Nigel Jones: thanks. Thanks. I understand. Now. Okay.
Nigel Jones: and that's that's all. I haven't done anything on it, since, though.
Jürgen Hemelt: would you think that there would be an option to use this plane. though? That's a starting point to like what we discussed to create
Jürgen Hemelt: a a standalone server. And this this specific standalone server, for instance, for only only one integration team. And then it's yeah. I wondered if it would be a useful exercise to go through to stimulate some ideas about that. I'm not saying literally, what's there now is necessarily the right approach. But it was the principle of saying.
Nigel Jones: Let's
Nigel Jones: use what is published by Nigeria as our kind of base catalogue of things that we can, we can use and consume and let's benefit from
Nigel Jones: the relationships are defining them grad or metadata, the dependencies, etc.
Nigel Jones: And then create something. Custom for what I want to run
Nigel Jones: within my repository as if I was a normal customer without having to recompile the base source code. So that was part of the thought process as well.
Nigel Jones: But, You know, I think there's more discussion around what you really want to do in that environment. And how do you refer to these packages? And then, ultimately you probably want a higher level of description. You might want to model
Nigel Jones: some of the things like the optional dependencies that we have. As well. You know, I want to create an integration server that uses Xtdb with postgres at the back end, you know, because that that brings in some other questions that are not reflected in dependencies that you need. You need a configuration, but you also need the
Nigel Jones: the postgres plugin for Xtgb, you probably need a Jdbc connector. I don't know. That's the case. I'm guessing it is that, and set it up.
Nigel Jones: But you need a number of pieces to put together. So it was just scratching on the surface of
Nigel Jones: of that concept of building externally, that's all. tiny baby step.
Jürgen Hemelt: And you also don't need the. For instance, the general score. If you use X to be.
Nigel Jones: That's right, that's right. That, in fact, the
Nigel Jones: the example I put in, I think, had just the core plus repository services.
Nigel Jones: But, for example, none of the omasses or the Graph Repository.
Nigel Jones: and I use the I used in memory. Actually, I think, for that example.
david radley: And the one thing that Mandy said was that she didn't like the platform being in the rest.
david radley: URL,
david radley: because it's a bit confusing to if you. She wants to differentiate between the top platform and usage and the sort of serving usage that we're envisaging.
Nigel Jones: Yeah, I think I think there's 2. I think there's 2 aspects there that one I think Manly didn't understand the context in which I was trying to do things. I think she was happy with Happy once I explained it to you. You know this is just a prototype. It's just an experiment. you know, and we've talked about the application, which is very separate from the platform. So I think she was okay with that. But the key point I think she was making was about terminology.
Nigel Jones: And you know we have this. We do have a bit of a cash in terminology into whether it's words words like server and platform, or whatever. So we just need to be really careful in
Nigel Jones: how things are explained and and how artifacts are named. I think that was her primary concern at the end.
Ljupcho Palashevski: So in regards to to terminology, I I this was a discuss on issue, I guess, right
david radley: this, this particular. It was an issue. But that was the springboard into the discussion. Really.
Ljupcho Palashevski: I think again from we are coming from in my kind of current state of mind. I that's why I would like to.
Ljupcho Palashevski: and again sorry about not only talking about it and not really making a progress really to separate. This is a standalone thing we call
Ljupcho Palashevski: I I I think that is completely a new spring blue application, and which will initialize with its own context. And then we should have the full control over.
Ljupcho Palashevski: indeed, the a new if we are talking about rest context, then it's completely new, and we will control that and definitely should not mentioned platform anywhere. And in regards to terminology, I think this should be the
Ljupcho Palashevski: a Max server application instead of a Microsoft platform. So that. That's, I think, as as simple as it it can be. It's just on my server platform now, the the spring good class, I think we should call it a Max server. That's it.
david radley: Yes, that's what that's what I think I did, or standalone server or something. So I think that made sense when we were talking about a health check endpoint.
david radley: I'm thinking
david radley: I can't remember it. Correct me if I'm wrong, but I'll all the calls and the like. They all have platform in them.
Ljupcho Palashevski: no, no, there is. There is the this separation of base pad. Basically, you have slash open metadata. And then that is the route for all admin calls, or you, if you will, more platform calls that are on not the capability rest endpoints, and then you have a slash server.
Ljupcho Palashevski: which is the the base of all access services and everything that is a capability Api. So it's really different on the base path.
Ljupcho Palashevski: So everything that is a platform related. I think it starts with slash, open metadata.
Ljupcho Palashevski: and then probably it has platform world depending on which Api. It is
Ljupcho Palashevski: admin services or
Ljupcho Palashevski: whatever it is, and then it has. When when you call, for example, when you call X service, it. It starts with server. I think
Ljupcho Palashevski: I can check that quickly.
david radley: I couldn't remember So presumably the host name and the port
david radley: really defines exactly what the platform would have been, and in our case would be, defining what the standalone server will be.
Ljupcho Palashevski: Yes, indeed, there is no platform mentioned, I think. but this is this open metadata starting context.
Ljupcho Palashevski: which I think mostly, then it refers to either admin services or platform services. But all of those calls have the
Ljupcho Palashevski: assumption that you are really running this open metadata platform, which can do something beyond the server.
Ljupcho Palashevski: and all of that, I think, is now out of scope for at least my again thinking process, because we should
Ljupcho Palashevski: do that prescri in a prescriptive way, as we discussed. Right? all of that should be already ready.
and then, once the service starts, you only expose this on the rest. Slash server
Ljupcho Palashevski: to to keep it really compatible with the on Mac Server that runs on the platform. I would still keep that slash server.
Ljupcho Palashevski: and then the rest.
Ljupcho Palashevski: but the the rest I mean by service as a user or whatever it is. I'm trying to find one example. Now.
david radley: yeah, make sense. I I'd forgotten exactly how well, so it's good to clarify.
Jürgen Hemelt: It's it's always open metadata, and then the services and then users and the
Ljupcho Palashevski: can I share quickly, because that's a but visually
Nigel Jones: I have this. I'll give you access.
Nigel Jones: There you go should have access.
Thanks
Ljupcho Palashevski: so quickly. Just Where is it? My postman? Here
Ljupcho Palashevski: it's been messed up, but
Ljupcho Palashevski: at least it will help us to
Ljupcho Palashevski: to share bit of this.
So
Ljupcho Palashevski: we have this platform services, for example, rest call to.
Ljupcho Palashevski: can you see my screen. Sorry I didn't check. Yes, cool. you see. for example, if you're addressing the platform
Ljupcho Palashevski: to say, see what service are active. This is just
Ljupcho Palashevski: that query call to to the platform. Then it starts with open metadata slash platform services similar to that
Ljupcho Palashevski: open metadata platform services to discover what is the status of a a server?
Ljupcho Palashevski: Then what are the services running? Not the service or services of a server running? It's still open metadata. So this is the all the time, the the base path, even even for the ones that are
Ljupcho Palashevski: like a capability for the platform. That's admin services. So you can see here, or this is still platform services. I should move this here, for example, this is admin services. So give me where this server is originating from things like, or I'm trying to configure a capability again through admin services. All, all of that is open metadata.
Ljupcho Palashevski: Now.
Ljupcho Palashevski: if I go to. This is all to configure specific type of service. That's how I have it. Them group like, if you're configuring
Ljupcho Palashevski: capability for a repository, or if you're configuring capability for access services that's still under this. for example.
Ljupcho Palashevski: I think. this is. Now, this is getting sorry. Sorry. Sorry.
Ljupcho Palashevski: Okay, too much.
Ljupcho Palashevski: this is, where is it?
Ljupcho Palashevski: I wanted to see? Yeah, still, on admin services. So if you're still configuring access services, you're still under this open my date. But if you're using access service, like, for example, to to search.
Ljupcho Palashevski: let's just go it to ask catalogue, and you start with servers, and you can see that pattern across all the access services. So this is data engine, for example, yeah, to create a file metadata.
Ljupcho Palashevski: it's again, this is a slightly differently map, but it's again slash service you can see in the variable here.
Ljupcho Palashevski: Same goes to all.
Ljupcho Palashevski: So point being all the capability Apis on the rest are always with the
Ljupcho Palashevski: slash servers as a server. Id, which I think the our new standard one application should be always
Ljupcho Palashevski: based on.
Ljupcho Palashevski: Like.
david radley: yeah, I just got forgotten that it's and I sing at just to check. I I don't think that any Omeg platform would issue a platform or admin call to another automatic platform
Nigel Jones: in the exchanges at all, I'm pretty sure. What about what about the deploy? Probably the under the deploy? It's not that useful in this environment. But there's, you know, when we do the configuration, we do the creation. That's a platform. Api. If I'm not mistaken, Nigel, we can check
Nigel Jones: yeah. And then that would be called on another platform.
Ljupcho Palashevski: Yeah, but that's right. That's a responsibility of a platform deploying config or exchanging configuration documents. So it's that old. So exactly, it's not relevant here. I'm just in the context, specifically of Davis question of whether one of what you would be I'm probably
david radley: It probably is. Okay. I was thinking, when you do an exchange when a new cohort member joins will be the actual
david radley: platform. They'll be. Actually they won't come from an application. We'll be in our own arrest. It won't be an O. Mass call. I think that's still an or us call it's it's almost as call, and that one is capability. Again. Here, for example, if I open this this is Sarah you want to. For example, a simple and fine entity. It's slash service open. The registration registration is not going through through Http.
Ljupcho Palashevski: Registration is is event based. And it's going through a
david radley: yeah. So that. And then what comes back in the registration is the rest endpoint, which would be an Rs for them. Which which is this? Which is one of those? Yeah, okay, I'll just checking that. I I can't. I just checking. There was nothing platform, me and all that other thing that raise
david radley: that that was just the only I I yeah, I'm I'm indeed, it's good to check. I think I'm I made this week analysis.
Ljupcho Palashevski: this is the only thing I did basically to to start thinking about the new implementation. So I think I can
Ljupcho Palashevski: definitely see? it's quite nice separation. because the the platform.
Ljupcho Palashevski: I mean, or the Admin class it. It's pretty much describing. You can see that platform services always. we can go in the code like if maybe that's better
Ljupcho Palashevski: the intelligence
delivery.
I told you guys.
Jürgen Hemelt: But don't you think that there is a a hard coded path somewhere in the client, libraries for the platform.
david radley: In some of the View Service, it says,
david radley: or in the view services when it's looking for the to like populate Rex and text, I think it issues
david radley: calls to the platform directly, to get the types and stuff is that
david radley: I think that there might be a platform, but or it might be a cervical
Ljupcho Palashevski: once again, which one Rex and text doing, what?
david radley: So if you go to if you go to view services. then go to say, Rex.
david radley: Rex view and then go to a server
david radley: source and then somewhere in there, when it is in. This is probably the handler actually
david radley: what it issues, the calls to get the So if you look for I don't know rest or post, or get
david radley: wherever I go to configure wherever it gets the
david radley: so it's not there, is it wherever it is. The rest of the call.
Ljupcho Palashevski: The rest call, I believe, is issued in the not in the handler, but probably
Ljupcho Palashevski: in the in the one of the
david radley: you rise.
Ljupcho Palashevski: So
david radley: so the question is, all those calls
david radley: and I'm thinking the one where it gets all the the which is probably it calls the text one. Actually. So it's probably actually in text low types. Here we go. So you get to load types
david radley: in the code on the right.
david radley: So
david radley: yeah. So presumably in this, get type explorer. And you could work down the code to see where actually issues, they
david radley: the the actual get to the to get wrecked. Where it? Where is it you call somewhere in there?
Ljupcho Palashevski: Okay, I I I think I think this this at least these 2.
Ljupcho Palashevski: so so it it really depends which one we are looking at. This is Rex. Right? So repository explorer really has this capability of discovering
Ljupcho Palashevski: or Mac Servers based on a platform
Ljupcho Palashevski: right? The platform is starting point, and then
Ljupcho Palashevski: it is sort of as the platform. What do you support from a perspective? And it's the same call I showed you.
david radley: So it's not that. It's because they're all hard coded out. They they because we hard code, the service and the platforms.
david radley: I'm just wondering when it gets the what's it called the I think it's does. It does not use the Api call David. If I remember correctly, this capability does not use the Api call, but actually, it's a configuration based, if you I think that's the case.
Ljupcho Palashevski: So the view Service will.
Ljupcho Palashevski: maybe maybe I'm mistaken.
david radley: Yeah, I don't. Yeah. I think it has to go there because the server might have different types. So it has to issue a call to get the types of the server. Yeah, that's fine. But that that that gets types is Omar S call. So that should be okay. Yeah, that's that. That, for sure is somewhere else call. But I was talking about discovering first before it gets there, the server, and that is, I think, the defined predefined. It's not going to like going going to the platform. Yeah. So it was just
david radley: if there was something around the the
Ljupcho Palashevski: I think I can't remember if it's an admin call, or or it's an call that just might be, I know for for a different use case. It's a on Rasco. So let's let's see going to repository services.
Ljupcho Palashevski: And just local it's basically implementing the same thing. But I want to go through. what is this
Ljupcho Palashevski: spring
Ljupcho Palashevski: resource? And then it's
Ljupcho Palashevski: get
Ljupcho Palashevski: at get
Ljupcho Palashevski: to boot type desk and get all types. This, this is the stuff that it's it's using basically types and and then not Tibet desk. So achieve it for time.
david radley: So against the only weird ones gonna be server offer, because that's creating admin calls isn't which we don't want them to be able to do.
Ljupcho Palashevski: Server author is tied to a platform capability.
david radley: So server author is something that we wouldn't really want to be doing for the cloud native stuff. So you probably want to switch off those because we'll be able to. You wouldn't make sense to run several. For in that case.
Ljupcho Palashevski: exactly. Because then the responsibility of all threeing service basically designing your landscape
Ljupcho Palashevski: is outside of a jury. Again, it it is responsibility of more of the either community operator or a physical admin that this defines this prescribes this environment
david radley: or deployment.
david radley: That's good. We found something that's a consideration. We just need to make sure.
david radley: but I guess we're what the way I was today in the The prototype was basically switch off all. I switch off all admin calls
david radley: so that you don't expose any endpoints, any admin endpoints, but you still have the
david radley: You still have the server side
david radley: implementation. So you call into the code and and and access the configuration
david radley: for the various. So because you still want to be able to access the configuration. But you don't need to be able to expose it in endpoints or update it via rest point arrest calls.
david radley: So as long as we just switch off the platform and switch off well, effectively
david radley: through the spring boot
david radley: when it's looking for the component scan. Just don't. We don't look for the for those spring endpoints. And we do include the new health points that we should be okay.
Ljupcho Palashevski: something like that. I was thinking, my plan was to basically just go and create this new spring booth application right? And do that in a in a best way we can. we can do it to leverage
Ljupcho Palashevski: it, Max. Then you may be a spring boot capabilities. So really a new spring boot application on Mac Server, which will Where is it? Among the operational resources
Ljupcho Palashevski: platform. So the replacement of this class will be a a a new in the new repo.
Ljupcho Palashevski: a spring, good application that will
david radley: do almost the same thing in a better and controlled way. What this operational services. I think you already did the part of it. I think I did the bits, and I switched off those. I'm pretty sure. I removed this from the composer. Scan as well. It might not be exactly right. What I did because of the
david radley: there was some code around. timing can when when things are starting.
david radley: and I'm hoping that moving all that's fine, and we're relying on the health
david radley: endpoint to to know when the things up or not.
david radley: so that we shouldn't end up having to cope with timing conditions to do with temporary deleted servers and things like that.
david radley: Oh, that was the risk code in that. that what you're doing, that.
david radley: So in the application, that is the
david radley: in the application that is the we, we got the new version of the actual top level. One. there was a load of extra logic to do with temporary restarting and things like this which I just removed
david radley: some I think that we can put there is. If there is some sort of consideration there, then we'll need to maybe reinstate some of it.
Ljupcho Palashevski: So what I wanted to do is you did. Really, when I have this new, the new shell application, or spring with starter.
Ljupcho Palashevski: really try to see
Ljupcho Palashevski: only from a from a kind of a building perspective. How can I instantiate from scratch.
Ljupcho Palashevski: step by step, components, if I need to write a new method instead of this, activate fine. So I really want to write a new startup sequence where all the subsystems, step by step, will be instantiated.
Ljupcho Palashevski: and then we will have this service state in control fully to also maybe expose it to a health check.
Ljupcho Palashevski: to to what level it is. Or we actually, we actually will
Ljupcho Palashevski: be more in control of the startup sequence, and then therefore
Ljupcho Palashevski: have a better
Ljupcho Palashevski: implement, a better health check, or those probes can provide more details as we instantiate the systems, and we progress with the startup sequence.
david radley: And I'm not chief. It's pretty much there, but it's You might need to tweak it with little bits. But it's there were this things like, maybe,
Ljupcho Palashevski: some of the existing classes are too entangled by what they're doing.
Ljupcho Palashevski: And assuming that the platform is there. So I I I want to
Ljupcho Palashevski: see if I can radically rewrite that code. So really not depend on this activate cold here. But right, new activation sequence. That was my plan.
Ljupcho Palashevski: And this is only on on assuming that the class, but is correct. And then, I think, when Nigel was talking about is to allow
Ljupcho Palashevski: better control over the class. But basically packaging when you're when you're building this to to not have extra things, but only things that you need.
Ljupcho Palashevski: and then they start a sequence. Of course we'll we'll have to have everything when it uses.
Ljupcho Palashevski: but then it will only
Ljupcho Palashevski: execute this. The the start subsystem, if it was declared, if it's not, was not declared, it means that most probably you don't want the classes available, and you will not trigger that instant instance. whatever subsystem instance creation.
Ljupcho Palashevski: then why are everything together?
Ljupcho Palashevski: So instead of it. It's it's slightly. I'm not sure if I have everything I need in place
Ljupcho Palashevski: similar to what you did in that poc. I I would like to Write it
Ljupcho Palashevski: outside of of this. basically not. Just look at what this is doing, and try to do everything manually, and just ignore
Ljupcho Palashevski: everything that is path platform to specific, basically skip that step or skip that configuration sequence, you know. Yeah, there was no effectively. The only bits, I think, was the re writing back
Ljupcho Palashevski: that safe server config on 2 to 8. But if yeah. But if we if we write this code ourselves, then we we we don't have to rely on what this sequence is doing. That's what I'm saying. Yes, basically, we'll not write the configuration. We don't need that
Ljupcho Palashevski: we have the configuration. It's what to do if you find that the service is already active.
david radley: so this is all of that stuff. I I wasn't sure.
Ljupcho Palashevski: But this makes sense for a context of a platform.
Ljupcho Palashevski: This, this, this doesn't make sense. I mean, you don't have the platform we reach can or cannot. have previous states. You you you really start from a new state.
david radley: Yes, so it shouldn't ever be. It shouldn't ever be started, because this is a new process that exactly this doesn't make. So that's what I'm saying. All the things that are platform
Ljupcho Palashevski: specific. I will call them like that
Ljupcho Palashevski: they don't make sense. Just keep. We will ignore, ignore, and get to a point where we
Ljupcho Palashevski: we. We have this new world for us, which is only a notion of a server.
Jürgen Hemelt: That's it. With that assumption there is the same value.
Jürgen Hemelt: and then off, and then you have the service that
Jürgen Hemelt: the instances that and what we we started
Jürgen Hemelt: once again. We don't need to handle this stuff
Jürgen Hemelt: because there is always only one server, and this is started, and if it is that then this is then a complete part of this. We started
Ljupcho Palashevski: exactly. Yeah, this. This is just the main state management on the platform level, basically recording or reporting the the status of this instance, which is the Java class instance. It's not the Java class instance. It's not the Java class instance. It's not a but for us. It's just one instance, and then of course, we want to still keep some, maybe state because of the probes we will see.
Ljupcho Palashevski: But indeed, it's one instance. And then that's that's that's how far it goes. We're going to have to maintain.
Ljupcho Palashevski: more than that as a state.
david radley: Yeah, I think it's I think what I did is pretty much what you want. There might be slight tweaks to it, but it depends on how much you want to refactor.
david radley: I want to have full control of the startup sequence. That's maybe
Ljupcho Palashevski: the only difference. I I think it's it's exactly the same where you started. I think we agreed to to go there. it's just I want a new class doing this.
Ljupcho Palashevski: you know, because then we have a standalone application which we did stand alone, control over the startup and in instantiation of systems. Not
Ljupcho Palashevski: not not really.
Ljupcho Palashevski: because the other thing is, I think what you ended up with is like, for example, not wanting to save the config in that, basically you you you try to. We I think you extended this class, or just to you
Ljupcho Palashevski: this something.
Ljupcho Palashevski: It's for me. It's it's a new startup sequence.
david radley: So you still need to set the operational configuration in the instance, because. as it goes forward this, this part of the configuration gets passed through to say, the access services and the like.
david radley: so that they know how to So I I came to the conclusion that you still needed the concept of a configuration.
david radley: But there's only one of them. what you could do, of course, is split. So here, you see this, if there.
david radley: if
david radley: this, the a phone through yeah, that one. So one way one thing you could do is say, well, we could just split it out for server, for metadata, access point or something.
david radley: and then just be, and this collapse it down like that. But at the moment we know that the logic is the same for so exactly how much you getting by splitting that out. I don't know. I know we were talking about how your model of for it.
david radley: that that they seem to be sort of we could probably get it working first, pretty much as is. and then work out what the next refinement is.
david radley: So, having the health having a rudimentary health check.
david radley: and having the ability to start single server on a platform, I think, proving that that can work with with no, with nothing else, I think would get you 70, 80% of the way there
david radley: and then the rest of it is gonna be optimizing jail files. It's going to be getting rid of superfluous code. It's gonna be maybe not exposing extra endpoints that you don't use, and things like that on a per basis. So you've got 70 80% with those 2 things, basically what Nigel did. a a rudimentary version of that, and also the the rudimentary version of the stand up.
david radley: I think the combination that gets you most of the way towards the cloud native and the rest is sort of
david radley: It's gonna be optimization, because you'll have the function
david radley: but it'll just be. It won't be as perfect as it would be if you obviously So I would get for something working cheap and cheerful, and then and then sort of iterate on what you need to do to make it
Ljupcho Palashevski: for me. This this this is just a matter of how do I call it?
Ljupcho Palashevski: I know that this delays something concrete to to have deliver something concrete as soon as possible, and that's probably what the what the intention of what you're saying is really.
Ljupcho Palashevski: as soon as you start changing for me is the little bit in my head at least other way around, because then I will actually see what is extra or a cloud native application.
Ljupcho Palashevski: So it's a little bit like you are trying to go from 1 point to somewhere, and I'm
Ljupcho Palashevski: completely trying to go from a different point to that somewhere, because, I really need to understand what is X-ray and and just don't don't have it basically slammed it down everything to bare minimum.
Ljupcho Palashevski: And it's it's also part of the really understanding.
Ljupcho Palashevski: all the the, the the
Ljupcho Palashevski: in a dependence here between this subsystem is, it's more more about really having that full control. And I understand. I mean, we basically, what you did is all almost done right. We just need to agree how we merge that. that change you, you, you you you figure out that it's a conflicting either it's a new method, or you know.
Ljupcho Palashevski: or extending and making the this same matter more
Ljupcho Palashevski: parameters. I don't know but
Ljupcho Palashevski: That's one option.
Ljupcho Palashevski: But I was more for this radical thing. I know it will delay the the end dates. But I think there is more value at the end.
Ljupcho Palashevski: considering I I don't think it's an either, or it's just what what you want. Let's let's let's work on both. I I would say
david radley: I would some, so I would. I mean I would personally deliver something that was cheap and cheerful, and then do the next. That's what I would suggest, and then do the next bit. But if you don't, if you want to go straight for the second part. That's fine as well.
david radley: It's just means you don't get something you cause you. You'll learn a lot by actually trying it, I think, because and have something working that's that's just in terms of
david radley: simple progression of changes.
Ljupcho Palashevski: that's that's what I would suggest. But it doesn't have to. That's a few. If you would really prefer to go straight for for for me, you know it's a bit not so. Maybe it's not back kind of only pragmatic. But it's also really
Ljupcho Palashevski: we thinking this like. I don't have to do anything with the platform. I'm writing my new application and trying to
Ljupcho Palashevski: really not adjust to the previous
Ljupcho Palashevski: designed, but
Ljupcho Palashevski: design it from scratch. It's more about that approach, like.
david radley: okay.
david radley: are you going to be bound? All of the It area architectural decisions around configuration and startup of and it's still, maybe we use because we know where the the the common point is. Actually it. It has to interoperate.
david radley: Yes, helpful.
Ljupcho Palashevski: That is for me. Whether you start with the platform or not, we agreed. I think, as a principle. There we need to interoperate. So everything
Ljupcho Palashevski: should work exactly the same, no matter how you started
Ljupcho Palashevski: the principal. And I want, I wanted to just do it from scratch basically start in a simple spring boot application that is going to have the same Api rest endpoints. Who is going to be able to instantiate the messaging connectors to I don't know register and all of that which is in the sequence here.
Ljupcho Palashevski: But do it step by step and also for that learning thing and and really thinking about hey? Now, I'm I'm writing a cloud native application. What can I do better me do I need to refactor some of those subsystems, do they? Are they good enough
Ljupcho Palashevski: things like that? But it's I agree that is something, maybe more at all or really
Ljupcho Palashevski: bigger change.
Ljupcho Palashevski: And I mean you.
Ljupcho Palashevski: what? What? What prevents us to merge a version of what you have already in a Pr
david radley: to be available so people can. I don't know. Use that to to put it in the new repository.
Ljupcho Palashevski: Yeah. But then we are replicating the that's I I that that new repository comes with the new kind of little bit, the redesign of this notion that it's a really server, and this one will still have platform and and a lot of things here
Nigel Jones: and there. So there's only one. It's just an experimental place, was my thoughts. I mean, how you want to use it, I guess, is up to you. But I thought it was more to support prototyping and really making any statement about.
Nigel Jones: because I think there's still a lot of things to work out, a lot of things to try independently. So that was the only reason I put it together, really.
Ljupcho Palashevski: So let's
Nigel Jones: we can continue like, is it? Now in the new rapport? I'm sorry I didn't. check that. The only thing is in the new repo is the the health check that I've played with.
Ljupcho Palashevski: so we may just merge that, or push that pr as a
david radley: It probably doesn't make sense to me to push it, as is
Nigel Jones: because what it does, it sets up and you.
david radley: And and so it has a new application. and it has a new so you probably didn't want that in corridor until we're really sure.
david radley: so it you could move all of the stuff over into the kind of native repo.
david radley: And what the only question is that one method that I added.
david radley: and we somehow
david radley: how keep that in the repo, or do we? Is it easier? And and should we put it into the call Repository?
david radley: So basically, it's a one class that is overlapping. I think it's this operational services. Do we copy that in the new rapport? And then we have the the new implementation for this.
Ljupcho Palashevski: I'm not sure the better. That is smartest thing to do, or we extend this class or something. So the the new spring, with application, we'll use that.
Ljupcho Palashevski: What we can do. Maybe we can have a a
Ljupcho Palashevski: I don't know a temporary version of this class somewhere.
Ljupcho Palashevski: I don't know. I don't know. This is what I I I was trying to avoid, not have this intermediate stand. Think about it, and then start thinking from scratch. But yeah, if you guys have a proposal how we can
Ljupcho Palashevski: get this intermediate state into the new repo. Let's do it.
Ljupcho Palashevski: But I will start writing that new application.
david radley: Okay, I I just need to Answer the door a minute and then I'll be back in a minute. Okay?
Jürgen Hemelt: Also, I have to leave early today.
Jürgen Hemelt: You guys any preferences
Jürgen Hemelt: you again. And
Jürgen Hemelt: I I don't have any preferences. But what I was thinking about is we we talked about this startup sequence.
Jürgen Hemelt: Do we have also to like, consider having another down sequence. Because when you're in a company, this environment.
Jürgen Hemelt: it always that you get signals.
Ljupcho Palashevski: Yes, yes. So the whole life cycle
Ljupcho Palashevski: of the new application will be managed in that new application in a way we want. In. In other words, so there is this activate. And I think this activate, or
Ljupcho Palashevski: I don't know how it's called the the Activate Survey, or something. So there is this sequence. We should we implement ourselves in a in a in a way.
Ljupcho Palashevski: how we would like to behave in a in the clouds or in a container orchestrated container.
Jürgen Hemelt: Yeah, that's fine.
Jürgen Hemelt: Okay. so I have to leave now
Ljupcho Palashevski: take care. I'll talk to you next week.
Jürgen Hemelt: Yeah, next week.
Jürgen Hemelt: Bye, bye.
david radley: I guess the interesting thing is,
david radley: shut down is going to be interesting because ideally, you'd want to do a
david radley: I don't know whether I think we have a soft and a hard shut down sort of concept. So if you
david radley: had like
david radley: then you'd want to do tell the cohort that you going away, and everything will be nice
david radley: But normally, in Kubernetes, potentially, what could happen is that you could just get chopped
david radley: and then so so I don't know whether we could do everything nice to do in that chopping world to to sort of somehow.
david radley: Tell the rest of the cohort that we've gone.
T. Taylan: I mean, we can assume that most of the time Kubernetes will chop this off, so the application should be able to revive after that. So I don't need to
T. Taylan: any. Do we need a clean shutdown on Kubernetes
Ljupcho Palashevski: the the regular one which is starting to stopping server is this one the activate running server instance or service instance, I think it is ever
Ljupcho Palashevski: which means that this is the the soft, I think, method which the registers or not, that does not de-regist, their server of a cohort, which is basically it's temporarily unavailable, which means either, for there there are more I see here.
Ljupcho Palashevski: But we need to dive a little bit. I don't know how these methods are mapped to the concepts, but I know the concept is that you can start and stop a server which should not impact the configuration of fathers, which is temporarily disabling this capability. But everything works.
Ljupcho Palashevski: And then
Ljupcho Palashevski: the the metadata exchange works in a way that
Ljupcho Palashevski: it. It assumes that server will be back, and nothing really changes. But then the the permanent is really a new state, and it's state it. It's a do register itself from a cohort, I think. So. That's a a conceptually different, basically it it removes the.
Ljupcho Palashevski: I think there are events sent the register events, and then it removes the registration previous made registration which can happen on a startup. And then the others are affected. Basically there you change a state on the cohort.
Ljupcho Palashevski: And then we will need to think, I think the regular one that's on to Kubernetes controls. Basically, the life cycle. Starting, stopping is nothing to do with the cohort.
Ljupcho Palashevski: It's basically it's life cycle of the instance. And one of these is that exactly.
Ljupcho Palashevski: But the other one.
Ljupcho Palashevski: we should. we should think about how we would like to to happen. And it's, I think, outside of of
Ljupcho Palashevski: Kubernetes control is basically on a different level. It's more of a
Ljupcho Palashevski: but you you you can. You can either. think of an you can do that from a platform existing on my platform. Because there is this.
Ljupcho Palashevski: I think. if I'm not mistaken, but we can check, or you can just send an event to the register, or it can be a server. Api call that you can if you want, like the capability Api calls it can be a I don't know. There is the server permanently, which will not kill the instance, but it will they register from a cohort. So it is.
Nigel Jones: You also remember it separate as well, because you've got replicas right of these service.
Nigel Jones: And just because one replica goes away. it doesn't mean the others are going away.
Nigel Jones: And the only way of understanding the difference between those 2 is with something that understands the current kind of topology as well as the users. Intent.
Ljupcho Palashevski: Yeah, exactly, but that that that those are 2 separate things. Right? So
Ljupcho Palashevski: so you you, your intent to the register from a coord is one thing, and then you should probably not forget that there are physical instances running, and you can maybe have some automation logic in the operator for this or not. Maybe it will be still a I I don't know. You know, some operational guides. If you did register. Then also I don't know. change the deployment something to scale down the replicas.
Ljupcho Palashevski: Things like that.
david radley: The I was thinking that we wouldn't be issued any admin calls to to at all. Apart from the health point, the health checkpoint.
david radley: So I was thinking that I'm not saying that that that's the way we should go. I'm just saying this can be completely outside of the Api. Of that
david radley: we could chop The Kubernetes will chop, in which case we don't go through any sort of nice same again. Well, because we we may not be going away, as you say. So maybe we just chop and chop and
david radley: If at some stage, if we have an operator that says we have 0. It wants 0 service, 0 replicas.
david radley: At that stage you could maybe bring up a process that would, with the with the appropriate configuration to d-register just for the purpose of d registry.
david radley: In that case, that that would, with exactly the same configuration that we have in the cohort store and all that stuff, so that that seems to be a a sort of mitigate more Kubernetes friendly way of doing it.
david radley: But you. You said if I understood correctly, you said to bring up new process. That will do the registration right? The registration right?
Ljupcho Palashevski: Yeah. Some sort of helper process that would with can be can be a new a helper class.
Ljupcho Palashevski: which you can start in a container, indeed.
Ljupcho Palashevski: just as yeah, indeed.
Nigel Jones: it's going to. And it's these used cases where we get into that, you know, it's not great to us, because it understands the events that are in the cluster, and can then do things that are
Nigel Jones: application specific
Nigel Jones: to, you know. Get you the behavior you want, and that's one example.
Nigel Jones: By the way in
Nigel Jones: you keeping us is normally, if it's a graceful shut down.
Nigel Jones: I think you have 15 s by default
Nigel Jones: to I think the Sync, 15 or something.
Nigel Jones: and you do have a really short period.
Nigel Jones: May help. It's always nice to do
Nigel Jones: that. You can. You've always got to be able to teach it with going down completely, and I do agree.
Nigel Jones: No, but that that's it. It's we we should not know. This is the the point is on Chip down there. Other things I agree with you for the cobalt removed. But let's say, for example, you haven't committed the read offset.
Nigel Jones: You know this. Things like that.
Nigel Jones: You might decide. It makes sense to if you can.
david radley: Yeah. Makes sense.
david radley: So the Kafka is a bit happier.
Nigel Jones: Well, to avoid responding to maybe a message twice or something? I I I don't know. There may be other examples, but it' be an optimization, not a man.
Ljupcho Palashevski: So the the graceful shut down, or signal shut down on a or mock server
Ljupcho Palashevski: we can think of as a as a extra. I think it's in the scope of the extra application or utility as well as the the registration.
Ljupcho Palashevski: because, if it's the the I don't know keel pot things like that, you will never get to this. The activation sequence or grace will shut down where you are also calling all the connectors to stop.
Nigel Jones: No, I mean, that's my point that
Nigel Jones: all you can do is a best step, but it's within the container itself.
Nigel Jones: All you can do is the best efforts by registering a shutdown hand left
and responding quickly, but think.
Nigel Jones: is there anything I could do
Nigel Jones: that is different then, to other components that are monitoring.
Nigel Jones: such as an upright T,
who may have the option to do?
Nigel Jones: Exactly. It's the long running.
Ljupcho Palashevski: Can you still hear me? I I think I heard some interaction.
Nigel Jones: I can hear you. okay,
Ljupcho Palashevski: yeah. Yeah. So so there, there, there's quite some aspects.
Ljupcho Palashevski: we need to map against what? What this extra. So for for the scope of the survey. We definitely, I think we all agree here. We are not going to provide Apis to manage the state in the cohort that should be a concern outside of the application instance, which will only start up
Ljupcho Palashevski: and we can implement a shut down, but that shut down can be only on a signal of a application. Jvm, shut down I, which which is the the regular, I I think. how is it called?
Ljupcho Palashevski: killing the the spring boot context, and and really trying to to bring everything down. We can register that on that event which is the event from a
Ljupcho Palashevski: spring booth, I think, and we can do it to our best to really
Ljupcho Palashevski: to everything we can for the application instance. That will not nothing
Ljupcho Palashevski: have to do that will not have to do with this. More of a
Ljupcho Palashevski: a concept of stopping or Max server or d registry which should be handled outside
Ljupcho Palashevski: later on, maybe automated, if we want to in operator or just a separate utility
Ljupcho Palashevski: for a manual interaction
Ljupcho Palashevski: to manage the cohort, basically, this is a. There is a need if we.
Ljupcho Palashevski: because the the all Max survey platform this
Ljupcho Palashevski: things with application instances or not really on Jvm level. But on this application level, where it instantiated or destroyed classes.
Ljupcho Palashevski: But then there was this, all other
Ljupcho Palashevski: more of a logical management of a
Ljupcho Palashevski: cohort
Ljupcho Palashevski: or cohort membership. Things like that which is still
Ljupcho Palashevski: still needs to be there. Just that. Now we don't have that Api both platform to
Ljupcho Palashevski: do it, and we should compensate. Maybe, if you are going fully without the platform, how you're going to, I think it should be open nice operational guidance on how that will be done
Ljupcho Palashevski: within you.
Ljupcho Palashevski: Application. But I think the important thing is, we are really scoping now what is important for the Jbm. And and container life cycle level. What is more of a
Ljupcho Palashevski: a life cycle, if you will in a in a court
david radley: sounds good.
Ljupcho Palashevski: Okay. I also have to go very quickly. I'm just summarizing
david radley: what?
david radley: I I don't think I'm going to actually be doing any actions. What? What do you want to go through the actions or
Ljupcho Palashevski: so in my agenda. I still I I have a lot of, and clarity as well.
Ljupcho Palashevski: but at least, if there is going to be time spent. I I I was planning to to this
Ljupcho Palashevski: from scratch
Ljupcho Palashevski: spring application
Ljupcho Palashevski: I can look into. I started looking into the Pr. David, but I didn't really dive into a lot of details. I will give you another chance to see whether we can first merge that, so we can have this.
Ljupcho Palashevski: I will. I will spend some time to understand that if I
Ljupcho Palashevski: don't figure it out in in terms of how we can put it in the code base. Then I will just continue with I will. I will. I can get back next week, for example, with the
david radley: with a concrete decision. If you want to talk about it, I'm happy to talk about like the details.
Ljupcho Palashevski: Okay, I always say, yes, I. The problem is, I never find time. But I I I will definitely. Now it's less so this all
Ljupcho Palashevski: a turbulence within the organization.
Ljupcho Palashevski: yeah, we're not doing regular stuff. So in the in that in between time. Well, I'm waiting. I'm I'm planning to
Ljupcho Palashevski: to spend time with this.
Ljupcho Palashevski: so I'll ping you for sure if I decide to go that way, or
Ljupcho Palashevski: let's just discuss it.
david radley: Yeah, sounds good.
Nigel Jones: Okay, yeah. The oxygen must have just dropped down. You just waiting to descend to 10,000 feet, and then you'll be okay.
Ljupcho Palashevski: Yeah.
Ljupcho Palashevski: Are you? Are you on a flight to Nigel?
Nigel Jones: No, I'm just walking.
Ljupcho Palashevski: Okay.
Ljupcho Palashevski: shall we slump here?
Nigel Jones: Yeah, sounds good. It was an interesting discussion. yeah, I just wish I could do more of it. But
Nigel Jones: that'd be no
Ljupcho Palashevski: okay till next week. Then
Ljupcho Palashevski: take care. Guys
david radley: talk to you.