Building Kubernetes Operator – Essential Components and Practical Guide

Introduction At the time of writing this blog, it seems the most popular topics in software engineering are Artificial Intelligence (AI) and Large Language Models (LLMs). LLMs are being proactively integrated into production systems for various purposes and goals, leading to the critical question: “How can we be certain the model performs well?” Traditional software is deterministic and can be tested with clear pass/fail decisions. On the other hand, LLMs are notorious for being non-deterministic, which is embedded in their very nature. Without getting into details on the underlying architecture or how they perceive the world, LLMs are known for their hallucinations, drifting away from their targeted task, being too agreeable, etc.  These types of problems force us to adopt a different approach when it comes to grading the quality of our LLM-dependent systems. We shift our perspective to evaluation, i.e., scoring the output by a predefined scale, usually [0,1], instead of the traditional binary pass/fail. Different frameworks try to provide the best solution, and the one that stands out amongst many is DeepEval. What is DeepEval? DeepEval is an open-source Python evaluation framework for LLMs. With the main goal being to unit test LLMs, DeepEval operates similarly to Pytest, along with the additional caveats attributed to LLMs. DeepEval supports end-to-end as well as component-level evaluation.  DeepEval’s approach to evaluation relies on the following components: Test Case - a single interaction between the user and the LLM. This is a container that includes data for a specific evaluation and consists of: input - the prompt sent to the LLM actual_output - the recorded LLM response expected_output - the gold standard response for comparison (if available) Metrics - definition of the scoring rule used to evaluate the Test Case. They may be: Deterministic - use fixed algorithms and formulae (ex., exact matches, regex, ROUGE/BLEU, LaBSE...) Non-deterministic (LLM-as-a-Judge) - use an independent powerful model to “read” and assess the output. Evaluation Dataset - collection of Test Cases grouped together Individual Test Case Evaluation Ideal evaluation workflow using DeepEval Basic evaluation through an example The problem We can imagine working on an LLM-based assistant that answers questions from retrieved documents. How do we verify whether the response is faithful to the source and is semantically correct? Setup pip install deepeval sentence-transformersexport OPENAI_API_KEY="your-api-key-here" The OpenAI key is needed for the LLM-as-a-judge metric, while deterministic metrics run without external calls.  The test case from deepeval.test_case import LLMTestCasetest_case = LLMTestCase( input="What is Varnish?", actual_output="Varnish is an HTTP reverse proxy that caches responses in RAM.", expected_output="Varnish is a caching HTTP reverse proxy that accelerates web applications.", retrieval_context=["Varnish is a HTTP reverse proxy and a caching server that ","speeds up web applications by caching HTTP responses." ]) Two metrics will be used for this test case. One is an LLM-as-a-Judge metric, and the other is a deterministic metric that will utilize LaBSE. The sentence-transformers package is used for LaBSE.  The retrieval_context is text fetched from a knowledge base that the LLM uses as source material when generating its answer. The retrieval_context is needed when we want to test the model against specific issues. (hallucinations, etc.) Language-agnostic BERT Sentence Embedding (LaBSE) is a multilingual sentence embedding model. It is used to compute the cosine similarity between the actual and expected outputs to evaluate semantic similarity. Metric 1: Faithfulness (LLM-as-a-Judge) LLM-as-a-Judge metrics use a secondary independent LLM to evaluate and score the primary model’s output. DeepEval has different built-in options: from deepeval.metrics import ( AnswerRelevancyMetric, # Does the output answer the question? FaithfulnessMetric, # Is the output grounded in the retrieval context? ContextualRelevancyMetric,# Is the retrieved context relevant to the question? HallucinationMetric, # Does the output contain fabricated claims not in the context? GEval, # Custom LLM-judged metric using criteria you define. The criteria is defined via a separate prompt.) For our use case, we will use the Faithfulness metric to check if the actual LLM output is grounded in the retrieval context. from deepeval.metrics import FaithfulnessMetricfaithfulness = FaithfulnessMetric(threshold=0.8) Because the evaluation itself is performed by an LLM, scores may slightly vary between runs. Metric 2: LaBSE semantic similarity (deterministic) from deepeval.metrics import BaseMetricfrom deepeval.test_case import LLMTestCasefrom sentence_transformers import SentenceTransformer, utilclass LaBSESimilarityMetric(BaseMetric): """Deterministic semantic similarity using LaBSE embeddings.""" def __init__(self, threshold: float = 0.7): self.threshold = threshold self.score = 0 self.success = False self.model = SentenceTransformer("sentence-transformers/LaBSE") async def a_measure(self, test_case: LLMTestCase, *args, **kwargs) -> float: return self.measure(test_case) def measure(self, test_case: LLMTestCase) -> float: embeddings = self.model.encode( [test_case.actual_output, test_case.expected_output], convert_to_tensor=True, ) self.score = util.cos_sim(embeddings[0], embeddings[1]).item() self.success = self.score >= self.threshold return self.score def is_successful(self) -> bool: return self.success @property def __name__(self): return "LaBSE Semantic Similarity" The class extends BaseMetric from DeepEval and implements: measure — encodes both actual_output and expected_output into embeddings, computes their cosine similarity, and tests it against the threshold. a_measure — async wrapper required by DeepEval's interface is_successful — returns whether the measurement passed the threshold. For the same inputs, LaBSE always produces the same embeddings, which follow with the same cosine similarity score. As a result, this metric is fully deterministic and reproducible. Running both metrics on one test case from deepeval import assert_testdef test_varnish_response(): assert_test(test_case, [ FaithfulnessMetric(threshold=0.8), # non-deterministic LaBSESimilarityMetric(threshold=0.7), # deterministic ]) Assuming the file containing the mentioned code is named test_varnish.py, the tests can be invoked through the following command: deepeval test run test_varnish.py Running the command produces the output: The test passes only if both metrics meet their thresholds. The faithfulness check ensures the output doesn't hallucinate beyond the context, while the LaBSE similarity check ensures it remains semantically close to the expected answer. Conclusion LLM-as-a-Judge can be used for free-form text where it is hard (or sometimes impossible) to define the correctness of an answer. (summaries, explanations, faithfulness, etc.) Deterministic metrics should be used for tasks with clearly defined correct outputs and where reproducible scores are needed. Another advantage of deterministic metrics is the avoidance of LLM-judge-related costs. In practice, the most effective evaluation suites combine both approaches. DeepEval provides a bridge between the familiar world of Pytest and the modern requirements of LLM testing. By combining LLM-judged metrics for subjective quality assessment with deterministic metrics for measurable correctness, it provides a complete evaluation toolkit that can be easily run from the CLI or a CI pipeline. References: Feng, F., Yang, Y., Cer, D., Arivazhagan, N., & Wang, W. (2020, July 3). Language-agnostic BERT sentence embedding. arXiv.org. https://arxiv.org/abs/2007.01852 DeepEval by Confident AI. (2026, April 5). Quick Introduction. https://deepeval.com/docs/getting-started LLM-as-a-Judge Metrics | Confident AI Docs. (n.d.). Confident AI Docs. https://www.confident-ai.com/docs/llm-evaluation/core-concepts/llm-as-a-judge Ip, J. (2025, October 10). LLM-as-a-Judge simply explained: The complete guide to run LLM evals at scale. Confident AI. https://www.confident-ai.com/blog/why-llm-as-a-judge-is-the-best-llm-evaluation-method DeepEval by Confident AI. (2026a, April 5). “Do it yourself” Metrics. https://deepeval.com/docs/metrics-custom Zheng, L., Chiang, W., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2023, June 9). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv.org. https://arxiv.org/abs/2306.05685

What is Streamlit? Working with tons of data can easily get overwhelming. Have you ever wanted to explore and visualize it quickly, without all the frontend setup? Streamlit has you covered. Streamlit is an open-source framework that enables engineers to create dynamic data web applications using Python without any frontend setup. There is no need to use HTML, CSS, JavaScript, or any complex web framework. Python knowledge is everything you need. To get started with Streamlit, simply follow the official installation guide. The Rerun Model - How Streamlit Actually Executes Coding using Streamlit is easy once you understand how it actually works. Streamlit applications are Python scripts that run from top to bottom and have a specific flow and architecture. Like any Python script, Streamlit apps execute statements sequentially from top to bottom. Every user interaction with the component triggers the entire script to be rerun. The need for this stems from Streamlit's simplicity: write apps the same way plain Python scripts are written. When coding with Streamlit, you don’t need to think in terms of callbacks and events, because each user interaction triggers a full rerun of the script to update the UI. Sessions - Preserving State Across Reruns A session is a single instance of the state of a running Streamlit application. Application state consists of the state of every UI component and is stored in-memory on the server as st.session_state. Every user session has its own st.session_state. For example, if we open our application in two tabs, each tab will have its own session. Since the script is rerun on each interaction, it is necessary to save the components' previous state to render them accurately on the next run. Streamlit components have an optional parameter key that, when set, points to the session_state key that holds the component's state, as shown in the example below. import streamlit as stst.radio(    label="Choose a movie genre",    options=[        "Comedy",        "Drama",        "Romance",    ],   key="radio_movie_genre")if st.session_state.radio_movie_genre == "Comedy":st.text("You have chosen Comedy")elif st.session_state.radio_movie_genre == "Drama":  st.text("You have chosen Drama")elif st.session_state.radio_movie_genre == "Romance":st.text("You have chosen Romance") The important thing to remember is that values in session state are only available in the current active session.  Caching - Avoiding Redundant Work The fact that the app is rerun frequently can cause significant time and memory overhead when loading data from the database or fetching it via an API call. On every rerun, the data would be fetched again. To prevent this, Streamlit provides a built-in caching mechanism. Cached values can be accessed by all users/tabs of the app, unlike a session, which can be accessed only by a single user/tab.  @st.cache_data Since the cached values are used by multiple sessions, modifying the data can cause a race condition. To ensure the original data state remains unchanged, Streamlit returns a copy of the cached data instead of the original. To cache the return value of the function that fetches the data, append the decorator @st.cache_data above its declaration. @st.cache_datadef fetch_the_data(param1:str, param2:str) -> List: # API call or DB access return data To determine whether to call the function or read the cached value, Streamlit checks the two conditions:  If the number, variable types, and values of the input parameters are the same  If the code in the function is the same But how does Streamlit actually check whether the input parameter values are the same? This is where hashing comes into play. The input values are hashed and stored so that, on subsequent calls, they can be compared with new values. On the first run, the function is called; on subsequent runs, if the conditions are met, the returned value is read from the cache, and a copy is created. By setting the ttl (time-to-live) of the cache decorator, the time during which the cache exists is determined. After that time, the cached value is no longer available, and the function will be executed again on a call. #The cached data will be stored for 3600s ( 1 hour )@st.cache_data(ttl=3600) def fetch_the_data(param1:str, param2:str) -> List: # API call or DB access return data @st.cache_resource For caching global resources such as ML models or database connections, which are typically expensive to create, Streamlit uses the @st.cache_resource decorator. Unlike @st.cache_data, the @st.cache_resource decorator does not create and return a copy of the cached value; instead, it returns the same shared instance (singleton) of the resource every time. That makes it a perfect fit for caching resources that are not meant to be duplicated. This helps avoid unnecessary memory usage, especially for large objects like ML models or database connections. However, because a cached resource is shared across all users, sessions, and reruns, it raises thread-safety concerns. This means that multiple users or threads can access the resource simultaneously and read or write to it concurrently. Because of this, it is important to ensure that cached resources are thread-safe. A thread-safe resource behaves correctly when multiple threads access and use it simultaneously, without producing inconsistent results.  In general, cached resources should be treated as read-only or designed to handle concurrent access. The example of usage is shown below: import streamlit as stimport psycopg2import pandas as [email protected]_resourcedef get_db_connection(): conn = psycopg2.connect( host="host", database="database", user="user", password="password" ) return conn# Function that fetches the data from dbdef fetch_data(query): conn = get_db_connection() # uses cached connection df = pd.read_sql_query(query, conn) return df The illustration below, from the official Streamlit documentation, represents, in a simple way, what can be cached using these two decorators from Streamlit. Summary / Key Takeaways Creating a modern, interactive UI to visualize and manage large datasets is both fun and easy with Streamlit. The diagram below shows how Streamlit works under the hood. From user interaction that triggers the script rerun, through session state restoration, to the execution of the script, with caching as a secret weapon for improving performance and reducing memory costs. Now that you understand how Streamlit handles reruns, state, and caching, you are all set to build more powerful Streamlit apps. These concepts are what make Streamlit seem simple, yet powerful and efficient in the background. References: Streamlit documentation.

What is Serenity Screenplay?   Screenplay is a test automation design pattern used to write readable, maintainable tests; it is part of Serenity BDD. Screenplay uses a model based on the Actors. Every Screenplay test has at least one actor (you can also have multiple actors). You can model scenarios involving multiple users, each with specific Abilities and Interactions. Besides Actors, other important components of Screenplay are Tasks, Interactions, Questions, and Abilities. Tasks and Actions are reusable across different test scenarios, helping minimize code duplication.   To use Screenplay, you will have to add these dependencies into pom.xml file:   <dependency>    <groupId>net.serenity-bdd</groupId>    <artifactId>serenity-screenplay</artifactId>    <version>4.2.34</version>         </dependency> <dependency>    <groupId>net.serenity-bdd</groupId>    <artifactId>serenity-screenplay-webdriver</artifactId>    <version>4.2.34</version> </dependency> Actors An actor represents a user who interacts with the application. Actors have abilities, such as the ability to interact with a web browser. They also perform tasks, interactions, and answer questions to check whether the feature behaves as we expect.   This is how we define an actor and give the actor the ability to interact with a web browser using Selenium WebDriver: final Actor actor = new Actor("User").can(BrowseTheWeb.with(getDriver()));   When writing code with Selenium, it can get a little messy; with a lot of code, it can become hard to read or maintain. For example, it can take a lot of time to figure out the best way to interact with a specific button on a page. With Screenplay, we use actors to perform interaction: Selenium example: getDriver().get("https://testingpage.com/login"); final WebElement username = getDriver().findElement(By.id("username")); username.sendKeys("John-Doe"); final WebElement password = getDriver().findElement(By.id("password")); password.sendKeys("pass123"); final WebElement loginButton = getDriver().findElement(By.id("login")); loginButton.click();   In this example, we can see: Selenium focuses on how you find an element and click on it or type into the field. Code repeats. Screenplay example:   actor.attemptsTo(  Open.url("https://testingpage.com/login"),        SendKeys.of("John-Doe").into(By.id("username")),        SendKeys.of("pass123").into(By.id("password")),        Click.on(Button.withText("Login")) ); OR You can create a separate CommonFields class where you will store common fields: public static final Target USERNAME = Target.the("Username field").located(By.id("username")); public static final Target PASSWORD = Target.the("Password field").located(By.id("password")); public static final Target LOGIN_BUTTON = Target.the("Login button").located(By.id("login"));   And then use it in a test:   actor.attemptsTo(        Open.url("https://examplepage.com/login"),        SendKeys.of("John-Doe").into(CommonFields.USERNAME),        SendKeys.of("pass123").into(CommonFields.PASSWORD),        Click.on(CommonFields.LOGIN_BUTTON) ); In this example, an actor performs a sequence of actions, which describes what the user does, not how it is done. It is readable, reusable, and maintainable. If an element changes, you only need to update it in one place. Tasks A task is a high-level action that an actor performs. It consists of smaller interactions like clicking, typing, etc. We can additionally create a task for login: public class LoginTask implements Task {    private final String username;    private final String password;   public LoginTask(final String username, final String password) {        this.username = username;        this.password = password;    }  public static LoginTask usingData(final String username, final String password) {        return new LoginTask(username, password);    }   @Override   public <T extends Actor> void performAs(T actor) {        actor.attemptsTo(                SendKeys.of(username).into(CommonFields.USERNAME),                SendKeys.of(password).into(CommonFields.PASSWORD),                Click.on(CommonFields.LOGIN_BUTTON)        );    } } And then use it in test:   actor.attemptsTo(        Open.url("https://examplepage.com/login"),        LoginTask.usingData("John-Doe", "pass123") );     In previous examples, we saw how to click on some element or type into the field. Next, we will see more basic interactions: Move mouse Move the mouse on a specific element (for example, you can use it to trigger a tooltip): actor.attemptsTo(MoveMouse.to(CommonFields.LOGIN_BUTTON)); Double click actor.attemptsTo(DoubleClick.on(target)); Right click actor.attemptsTo(RightClick.on(target)); Enter value This will enter a value into a field, first it will wait until the field is enabled, and then clear the field of any current values, before entering the specified value: actor.attemptsTo(Enter.theValue("John-Doe").into(CommonFields.USERNAME)); Upload file Files can be placed in a hard-coded location or stored on the classpath, which is a better solution:   actor.attemptsTo(Upload.theFile(Path.of("test/data/api/files/" + fileName +".extension")).to(uploadField)); Questions When an actor wants to get a specific piece of information from the system, it can be done using a Question. Questions are queries about the application's state. Actors ask questions using the method askFor().   Some of the common Question classes are: Text.of()   final String text = actor.asksFor(Text.of(TARGET_ELEMENT));   This will extract the text of a target element and return the value that you can use for assertion.   When working with table, we can extract data from cells:   final List<String> tableHeaders = actor.asksFor(Text.ofEach(HEADERS)); final List<List<String>> tableRows = actor.asksFor(Text.ofEach(ROWS)); isVisible() boolean isVisible = actor.asksFor(Visibility.of(TARGET_ELEMENT));   This checks whether a UI element is displayed. isDisabled() boolean isDisabled = actor.asksFor(Disabled.of(TARGET_ELEMENT));   This will check whether an element is disabled. Presence.of() boolean isPresent = actor.asksFor(Presence.of(TARGET_ELEMENT)); This checks whether the element is present in the DOM. Conclusion With Serenity Screenplay, you can write clean and maintainable automation tests using Actors, Abilities, Tasks, and Questions. Instead of just writing a bunch of code with Screenplay, your automation code will be easy to read and understand for both QA and developers. It is also important to know that Screenplay doesn’t replace Selenium; they work together. Selenium is the engine that communicates with the browser, and Screenplay is the pattern that makes your tests feel like a story.