Can't connect to dockerized kafka (used wurstmeister/kafka-docker) from outside the server

See also questions close to this topic

• best way to add a spring boot web application with JSPs to a docker container?

  so i have a spring boot web application, it uses JSPs, and im supposed to put it in a container .
  my question is what is the best way ? ive tried to copy the project and then run it there using a wmvn like so : dockerfile:

  FROM openjdk:8-jdk-alpine
  ADD . /input-webapp
  WORKDIR /input-webapp
  EXPOSE 8080:8080
  ENTRYPOINT ./mvnw spring-boot:run
  

  which works, but it takes a long time getting the dependencies and feels messy .
  

  and ive tried to package it into a jar, and then copy the jar only, and run it : dockerfile:

  FROM openjdk:8-jdk-alpine
  ADD target/input-webapp-0.0.1-SNAPSHOT.jar input-webapp-0.0.1-SNAPSHOT.jar
  EXPOSE 8080:8080
  ENTRYPOINT ["java","-jar","input-webapp-0.0.1-SNAPSHOT.jar"]
  

  but this way it cant see the JSPs, or at least i think this is the problem, as i get a 404.
  

  so is there a better way ? can i copy the jsps plus the jar to make it work? thanks
• build spring boot (mvnw) with docker can not use cache

  Spring Boot Docker Experimental Features Docker 18.06 comes with some “experimental” features, including a way to cache build dependencies. To switch them on, you need a flag in the daemon (dockerd) and an environment variable when you run the client. With the experimental features, you get different output on the console, but you can see that a Maven build now only takes a few seconds instead of minutes, provided the cache is warm.

  my dockerfile can not use cache.

  dockerfile

  # syntax=docker/dockerfile:experimental
  FROM openjdk:8-jdk-alpine as build
  WORKDIR /workspace/app
  
  COPY mvnw .
  COPY .mvn .mvn
  COPY pom.xml .
  COPY src src
  
  RUN --mount=type=cache,target=/root/.m2 ./mvnw install -DskipTests -s .mvn/wrapper/settings.xml
  RUN mkdir -p target/extracted && java -Djarmode=layertools -jar target/*.jar extract --destination target/extracted
  
  FROM openjdk:8-jre-alpine
  ENV TZ Asia/Shanghai
  RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
  RUN addgroup -S spring && adduser -S spring -G spring
  USER spring:spring
  
  ARG EXTRACTED=/workspace/app/target/extracted
  ARG JAVA_OPTS="-Xmx100m -Xms100m"
  
  COPY --from=build ${EXTRACTED}/dependencies/ ./
  COPY --from=build ${EXTRACTED}/spring-boot-loader/ ./
  COPY --from=build ${EXTRACTED}/snapshot-dependencies/ ./
  COPY --from=build ${EXTRACTED}/application/ ./
  
  ENTRYPOINT ["sh", "-c","java ${JAVA_OPTS} org.springframework.boot.loader.JarLauncher"]
  

  run shell

  DOCKER_BUILDKIT=1 docker build -t org/spring-boot .
  

  every time use many minutes
• PyQT5 doesn't work on docker ImportError: libsmime3.so: cannot open shared object file: No such file or directory

  I have a Dockerfile with PyQT installed like below

  FROM ubuntu:20.04
  ENV DEBIAN_FRONTEND=noninteractive
  RUN adduser --quiet --disabled-password qtuser && usermod -a -G audio qtuser
  RUN apt-get update -y \
      && apt-get install alsa -y \
      && apt-get install -y python3-pyqt5 \
      && apt-get install python3-pip -y && \
      pip3 install pyqtwebengine
  WORKDIR /htmltopdf
  

  I built my image like this docker build -t html-to-pdf .

  Then I ran my image like this docker run --rm -v "$(pwd)":/htmltopdf -u qtuser -it html-to-pdf python3 htmlToPdfnew.py --url https://www.w3schools.com/howto/howto_css_register_form.asp

  But I'm getting below error

  Traceback (most recent call last):
    File "htmlToPdfnew.py", line 2, in <module>
      from PyQt5 import QtWidgets, QtWebEngineWidgets
  ImportError: libsmime3.so: cannot open shared object file: No such file or directory
  

  I do NOT get that error in my PC.

  below is my python code

  import sys
  from PyQt5 import QtWidgets, QtWebEngineWidgets
  from PyQt5.QtCore import QUrl, QTimer
  from PyQt5.QtGui import QPageLayout, QPageSize
  from PyQt5.QtWidgets import QApplication
  import argparse
  
  
  def main():
      url = ''
      parser = argparse.ArgumentParser(description="Just an example", formatter_class=argparse.ArgumentDefaultsHelpFormatter)
      parser.add_argument("--url", help="Type url")
      args = parser.parse_args()
      config = vars(args)
      url = config['url']
  
  
      app = QtWidgets.QApplication(sys.argv)
      loader = QtWebEngineWidgets.QWebEngineView()
      loader.setZoomFactor(1)
      layout = QPageLayout()
      layout.setPageSize(QPageSize(QPageSize.A4Extra))
      layout.setOrientation(QPageLayout.Portrait)
      loader.load(QUrl(url))
      loader.page().pdfPrintingFinished.connect(lambda *args: QApplication.exit())
  
      def emit_pdf(finished):
          QTimer.singleShot(2000, lambda: loader.page().printToPdf("test.pdf", pageLayout=layout))
  
      loader.loadFinished.connect(emit_pdf)
      sys.exit(app.exec_())
  
  if __name__ == '__main__':
      main()
  

  so how do I resolve below error ?

  Traceback (most recent call last):
    File "htmlToPdfnew.py", line 2, in <module>
      from PyQt5 import QtWidgets, QtWebEngineWidgets
  ImportError: libsmime3.so: cannot open shared object file: No such file or directory
  
• kafka - How to read and process messages in a fault tolerant way?

  I am new to Kafka, and I am trying to understand how a Kafka Consumer can consume and process a message from a Kafka topic without losing messages if the consumer fails in between.

  For Example:

  There is a kafka topic called cart_checkout which just holds an event when the user successfully checks out their cart. All it will do is get an event and send an email to the user that the their items are checked out.

  This is the happy path:

  1. Consumer gets an event from the topic.
  2. Read offset is committed to Kafka.
  3. Consumer calls app specific function to send email.
  4. Repeat.

  What happens if the app fails during Step 3 ?

  If the consumer starts up then it will miss an event, am I right ? (Since the read message was committed)

  The consumer can rewind but how will it know to rewind ?

  RabbitMQ solution:

  It seems easier to solve in RabbitMQ since we can control the ACKs to a message. But in Kafka, if we commit the read-offset we lose the message when the app restarts, if we don't commit the read-offset then the same message is sent to multiple consumers.

  What is the best possible way to deal with this problem ?
• Kafka unable to connect to Zookeeper in docker-compose.yml

  I am using the Zookeeper and Kafka images provided by the Debezium project in a docker-compose.yml file. When I bring up the containers, I see a lot of error logs in both the Zookeeper and Kafka containers.

  Zookeeper log:

  zookeeper_1  | 2022-05-06 17:28:50,926 - WARN  [NIOWorkerThread-10:NIOServerCnxn@370] - Exception causing close of session 0x0: Unreasonable length = 308375649
  

  Kafka log:

  2022-05-06 17:28:50,975 - ERROR [main:TopicCommand$@44] - org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics
  

  I do not know what is causing these errors or how to resolve them.

  My docker-compose.yml file and links to the Docker images are below.

  docker-compose.yml

  version: '3.3'
  
  services:
  
    zookeeper:
      hostname: zookeeper
      image: debezium/zookeeper:1.8
      ports:
        - 2181:2181
        - 2888:2888
        - 3888:3888
  
    kafka:
      hostname: kafka
      image: debezium/kafka:1.8
      ports:
        - 9092:9092
      environment:
        - ZOOKEEPER_CONNECT=zookeeper:2181
        - CREATE_TOPICS=kafka_connect_config:1:1:compact,kafka_connect_offsets:1:1:compact,kafka_connect_status:1:1:compact
      depends_on:
        - zookeeper
  

  Debezium Kafka
  Debezium Zookeeper

  If it helps, I am on an Ubuntu machine.
• Error while Upgrading Confluent Connector for S3 Sink

  I am currently working on a project where we have a already setup system with Confluent sink connectors for s3 and Elasticsearch. Current Version which we are using are :-

  Kafka Version :- 2.4.0
  Confluent Connector for S3 :- 5.3.1
  Confluent Connector for Elasticsearch :- 5.3.1
  

  The whole setup helps us to get data from kafka topic to sink(s3 and elastic). We are in process of upgrading the whole setup, though the goal is to get to the latest version but because of some technical issues we have to gradually upgrade the version. Now we are trying to upgrade to

  Kafka :- 2.5.0
  Confluent S3 sink :- 5.5.1
  Confluent Elastic sink connector:- 5.5.1
  

  This whole setup is deployed in Kubernetes, while deploying the upgraded image i am seeing below error

  java.lang.abstractmethoderror: receiver class com.sun.jersey.api.uri.uribuilderimpl does not define or inherit an implementation of the resolved method 'abstract javax.ws.rs.core.uribuilder uri(java.lang.string)' of abstract class javax.ws.rs.core.uribuilder.
  

  This is causing the connector to fail and exit. Am I missing out something here ??? Any one in community has encountered this issue ?? Any help to resolve this issue is greatly appreciated.
• how do i store maven dependencies inside docker image using a Dockerfile

  so im making a spring boot application that im supposed to put in and run from a docker container, and i would like to build the whole image using a docker file.
  im using this dockerFile:

  FROM openjdk:8-jdk-alpine
  ADD . /analytics-service
  WORKDIR /analytics-service
  ENTRYPOINT ./mvnw spring-boot:run 
  

  when i create the image it just copies the files, and only after i run it, it starts downloading all the maven dependencies. which takes a while, given that i will be running a few containers. so how do i do it ? i want it to get all the dependencies when the image is created, so when i create a container it doesnt start downloading.
• Uncaught Error: Class 'Service\UsersService' not found

  I'm trying to run my php-based application, which connects to a mysql DB, launched through docker-compose.

  i get this message when i connect to localhost:

  Fatal error: Uncaught Error: Class 'Service\UsersService' not found in /var/www/html/dic/users.php:3 Stack trace: #0 /var/www/html/web/index.php(3): require() #1 {main} thrown in /var/www/html/dic/users.php on line 3

  user.php

  <?php
  
  return new Service\UsersService(
      require "../config/db-connection.php"
  );
  

  UsersService.php

  <?php
  
  namespace Service;
  
  use Entity\User;
  use PDO;
  
  class UsersService
  {
      protected $db;
  
      public function __construct(PDO $db)
      {
          $this->db = $db;
      }
  
      public function getLastJoined(): array
      {
          return $this->db->query(
              "SELECT id, joined, name FROM user ORDER BY joined DESC LIMIT 10"
          )->fetchAll(PDO::FETCH_CLASS, User::class);
      }
  
      public function getById(string $id): ?User
      {
          $user = $this->db->query("SELECT id, joined, name FROM user WHERE id = " . $this->db->quote($id))->fetchObject(User::class);
  
          return ($user === false) ? null: $user;
      }
  }
  

  index.php

  <?php
  
  $lastJoinedUsers = (require "../dic/users.php")->getLastJoined();
  
  switch (require "../dic/negotiated_format.php") {
      case "text/html":
          (new Views\Layout(
              "Twitter - Newcomers", new Views\Users\Listing($lastJoinedUsers), true
          ))();
          exit;
  
      case "application/json":
          header("Content-Type: application/json");
          echo json_encode($lastJoinedUsers);
          exit;
  }
  
  http_response_code(406);
  

  I appreciate any help!!
• Docker compose specified port not mapping to connect to the health check endpoint

  I am a newbie to docker. This is the layout I am working with. I have 2 projects in the docker compose file. A dotnet6 web api and a mvc web app. In the web api startup.cs I have the following code to get health statuses using AspNetCore.HealthChecks.UI, If I browse to the endpoint I get a json object of health check values (this part works)

  app.UseEndpoints(endpoints =>
     {
      endpoints.MapControllers();
      endpoints.MapHealthChecks("/hc", new HealthCheckOptions
        {
          Predicate = _ => true,
          ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
        });
    });
  

  I have a WebStatus .Net core MVC app whose job is to display health stats using the healthcheck UI. The appsetting for this is:

  {
    "HealthChecks-UI": {
      "HealthChecks": [
        {
          "Name": "TestApp API Health Check",
          "Uri": "https://localhost:7100/hc"
        },
        {
          "Name": "TestApp API Health Check (Local)",
          "Uri": "https://localhost:7141/hc"
        }
      ],
      "Webhooks": [
        {
          "Name": "",
          "Uri": "",
          "Payload": "",
          "RestoredPayload": ""
        }
      ],
      "EvaluationTimeOnSeconds": 10,
      "MinimumSecondsBetweenFailureNotifications": 60
    },
    "Logging": {
      "LogLevel": {
        "Default": "Information",
        "Microsoft": "Warning",
        "Microsoft.Hosting.Lifetime": "Information"
      }
    },
    "AllowedHosts": "*"
  }
  
  

  And the docker compose file is as shown below

  version: '3.4'
  
  services:
    testapp.api:
      environment:
        - ASPNETCORE_ENVIRONMENT=Development
        - ASPNETCORE_URLS=https://+:443;http://+:80
      ports:
        - "8005:80"
        - "7100:443"
      volumes:
        - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
        - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
  
    webstatus:
      environment:
        - ASPNETCORE_ENVIRONMENT=Development
        - ASPNETCORE_URLS=https://+:443;http://+:80
        - HealthChecksUI__HealthChecks__0__Name=Test App API Health Check
        - HealthChecksUI__HealthChecks__0__Uri=https://testapp.api/hc
      ports:
        - "8006:80"
        - "7101:443"
      depends_on:
        - testapp.api
      volumes:
        - ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
        - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
  
  

  If I start both the projects without docker, everything works, the WebStatus app shows the health status of the api as expected! However it doesn't work when I use docker compose to run this. The url I use to get to the webstatus app is https://localhost:7101/healthchecks-ui#/healthchecks