Related questions

How WSO2 API manager implements microservices service discovery and service registry feature

2018-05-16 06:09 Java Techie imported from Stackoverflow

How WSO2 API manager implements microservices service discovery and service registry feature

1 answer

See also questions close to this topic

  • WSO2 APIM constraint violation CON_APP_KEY

    Sometimes, I encounter some problem to persist my new generated token in MySQL database.
    This problem is similar to this post or this.
    Unfortunately even with <PoolSize>0</PoolSize> in identity.xml, I have sometimes the problem.
    When it arrives, if I check the database, I notice the column TIME_CREATED is exactly equaled for the two last tokens. One is ACTIVE and the other INACTIVE. If I delete the active token everything work again. Occasionally, I can do it manually, but it isn't a real solution.
    On the first post, I saw the response of @Sjaak but don't find MySQL script which updates the column TIME_CREATED

    Thanks in advance for your help.

  • MQTT Oauth2 authentication against WSO2 Message broker

    I want to get a MQTT broker to authenticate with an OAUTH2 server.

    I am coding the MQTT client in Python 3 using Paho.

    I have succesfullly connected to the IS to obtain the access token.

    I have succeesfully sent json messages through the MQTT Message Broker.

    I have configured WSO2 Message Broker and Identity Server to work together following the oficial instructions.

    When I try to connect with the authentication information I cannot get to send correctly the acces token to be processed. 

    client = mqtt.Client("oauthmqttcli") 
client.connect(broker_address, port=1883) #connect to broker
client.username_pw_set(access_token)
client.publish("test","{'a':'b'}")

    In the logs I see this:

    TID: [] [] [2018-09-21 10:56:33,019] ERROR {org.dna.mqtt.wso2.MqttLogExceptionHandler} -  ValueEvent exception occurred on disruptor. {org.dna.mqtt.wso2.MqttLogExceptionHandler}
java.lang.NullPointerException
        at org.dna.mqtt.moquette.messaging.spi.impl.ProtocolProcessor.processPublish(ProtocolProcessor.java:390)
        at org.dna.mqtt.moquette.messaging.spi.impl.SimpleMessaging.onEvent(SimpleMessaging.java:171)
        at org.dna.mqtt.moquette.messaging.spi.impl.SimpleMessaging.onEvent(SimpleMessaging.java:47)
        at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
TID: [] [] [2018-09-21 10:56:33,026] ERROR {org.dna.mqtt.moquette.server.netty.metrics.MessageMetricsHandler} -  Connection reset by peer {org.dna.mqtt.moquette.server.netty.metrics.MessageMetricsHandler}
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
        at sun.nio.ch.IOUtil.read(IOUtil.java:192)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
        at io.netty.buffer.UnpooledUnsafeDirectByteBuf.setBytes(UnpooledUnsafeDirectByteBuf.java:446)
        at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881)
        at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:225)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:748)
TID: [] [] [2018-09-21 10:56:33,027]  INFO {org.dna.mqtt.moquette.messaging.spi.impl.ProtocolProcessor} -  Lost connection with client oauthmqttcli {org.dna.mqtt.moquette.messaging.spi.impl.ProtocolProcessor}
TID: [] [] [2018-09-21 10:56:33,027]  WARN {org.dna.mqtt.moquette.messaging.spi.impl.ProtocolProcessor} -  MQTTAuthorizationSubject for client ID oauthmqttcli is not removed since the entry does not exist {org.dna.mqtt.moquette.messaging.spi.impl.ProtocolProcessor}

    How can I send the access token to the server to be accessed? Where is the Message Broker searching for the access token?

  • WSO2 Identity Server with external GLUU OpenId Connect IDP - Authentication method client_secret_basic vs client_secret_post

    I’m testing wso2 Identity server with an external OpenID Connect IDP. The external IDP is GLUU.

    In my test case, I have three components:

    • a simple web app deployed on localhost:8099
    • a wso2 Identity Server deployed on localhost:9443 (with its default ports)
    • a GLUU server deployed on localhost (with its default ports)

    The flow I want to obtain is an OpenId Connect authorization_code, my "simple web app" (configured like a Service Provider in wso2is) requires a code to wso2is, are redirected to a GLUU login page, get the authorization code and exchanges the authorization code for an access token.

    The "simple web app" is registered as a Service Provider in wso2is (with the name "Service Provider Test") with an inbound authentication configuration of type "OAuth/OpenId Connect Configuration." In this configuration I have set the callback URL of my "simple web app," and the client secret and client id used by my simple web app to obtain the access token.

    This "Service Provider Test" in the section "Local & Outbound Authentication Configuration" is set to Federated Authentication, with "My Gluu Test IDP."

    My Gluu Test IDP is the name of an Identity Provider I have added to wso2is's "service providers." I have uploaded the Gluu's certificate to "My Gluu Test IDP."

    In the section "Federated Authenticators" of "My Gluu Test IDP" I have set up an "OAuth/OpenId Connect Configuration" with the information of the GLUU IDP (client id, secret, token and auth endpoints) and with this URL callback: https://localhost:9443/commonauth

    This was the premise.

    I try this scenario the first time and this is was I get:

    1. Calling wso2is authorization endpoint from my "simple web app"
    2. Redirect to GLUU login page
    3. Back to my "simple web app" callback uri with an error instead of the authorization code: {error_description=Authentication required, state=OX86oF, error=login_required, session_state=70b94648800ea5fa1e2476570ee9678225eaccc4deecea18ca9db022b7e1697c.fF2IdhJSG3aTE_TIsyGlYg}

    The wso2is log related to this error is:

    TID: [-1234] [] [2018-09-21 10:33:10,529] ERROR {org.wso2.carbon.identity.application.authentication.framework.handler.step.impl.DefaultStepHandler} - invalid_client, Client authentication failed (e.g. unknown client, no client authentication included, or unsupported authentication method). The authorization server MAY return an HTTP 401 (Unauthorized) status code to indicate which HTTP authentication schemes are supported. If the client attempted to authenticate via the Authorization request header field, the authorization server MUST respond with an HTTP 401 (Unauthorized) status code and include the WWW-Authenticate response header field matching the authentication scheme used by the client. org.wso2.carbon.identity.application.authentication.framework.exception.AuthenticationFailedException: invalid_client, Client authentication failed (e.g. unknown client, no client authentication included, or unsupported authentication method). The authorization server MAY return an HTTP 401 (Unauthorized) status code to indicate which HTTP authentication schemes are supported. If the client attempted to authenticate via the Authorization request header field, the authorization server MUST respond with an HTTP 401 (Unauthorized) status code and include the WWW-Authenticate response header field matching the authentication scheme used by the client. at org.wso2.carbon.identity.application.authenticator.oidc.OpenIDConnectAuthenticator.getOauthResponse(OpenIDConnectAuthenticator.java:585) at org.wso2.carbon.identity.application.authenticator.oidc.OpenIDConnectAuthenticator.processAuthenticationResponse(OpenIDConnectAuthenticator.java:370) at org.wso2.carbon.identity.application.authentication.framework.AbstractApplicationAuthenticator.process(AbstractApplicationAuthenticator.java:72) at org.wso2.carbon.identity.application.authentication.framework.handler.step.impl.DefaultStepHandler.doAuthentication(DefaultStepHandler.java:488) at org.wso2.carbon.identity.application.authentication.framework.handler.step.impl.DefaultStepHandler.handleResponse(DefaultStepHandler.java:462) at org.wso2.carbon.identity.application.authentication.framework.handler.step.impl.DefaultStepHandler.handle(DefaultStepHandler.java:165) at org.wso2.carbon.identity.application.authentication.framework.handler.sequence.impl.DefaultStepBasedSequenceHandler.handle(DefaultStepBasedSequenceHandler.java:176) at org.wso2.carbon.identity.application.authentication.framework.handler.sequence.impl.GraphBasedSequenceHandler.handle(GraphBasedSequenceHandler.java:63) at org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultAuthenticationRequestHandler.handle(DefaultAuthenticationRequestHandler.java:131) at org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultRequestCoordinator.handle(DefaultRequestCoordinator.java:157) at org.wso2.carbon.identity.application.authentication.framework.servlet.CommonAuthenticationServlet.doPost(CommonAuthenticationServlet.java:53) at org.wso2.carbon.identity.application.authentication.framework.servlet.CommonAuthenticationServlet.doGet(CommonAuthenticationServlet.java:43) at javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:37) at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:60) at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.wso2.carbon.identity.captcha.filter.CaptchaFilter.doFilter(CaptchaFilter.java:76) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.owasp.csrfguard.CsrfGuardFilter.doFilter(CsrfGuardFilter.java:72) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:80) at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:91) at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:60) at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99) at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57) at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62) at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962) at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1115) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1775) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1734) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: OAuthProblemException{error='invalid_client', description='Client authentication failed (e.g. unknown client, no client authentication included, or unsupported authentication method). The authorization server MAY return an HTTP 401 (Unauthorized) status code to indicate which HTTP authentication schemes are supported. If the client attempted to authenticate via the Authorization request header field, the authorization server MUST respond with an HTTP 401 (Unauthorized) status code, and include the WWW-Authenticate response header field matching the authentication scheme used by the client.', uri='null', state='null', scope='null', redirectUri='null', responseStatus=0, parameters={}} at org.apache.oltu.oauth2.common.exception.OAuthProblemException.error(OAuthProblemException.java:59) at org.apache.oltu.oauth2.client.validator.OAuthClientValidator.validateErrorResponse(OAuthClientValidator.java:63) at org.apache.oltu.oauth2.client.validator.OAuthClientValidator.validate(OAuthClientValidator.java:48) at org.apache.oltu.oauth2.client.response.OAuthClientResponse.validate(OAuthClientResponse.java:64) at org.apache.oltu.oauth2.client.response.OAuthClientResponse.init(OAuthClientResponse.java:59) at org.apache.oltu.oauth2.client.response.OAuthAccessTokenResponse.init(OAuthAccessTokenResponse.java:52) at org.apache.oltu.oauth2.client.response.OAuthClientResponseFactory.createCustomResponse(OAuthClientResponseFactory.java:60) at org.apache.oltu.oauth2.client.URLConnectionClient.execute(URLConnectionClient.java:111) at org.apache.oltu.oauth2.client.OAuthClient.accessToken(OAuthClient.java:65) at org.apache.oltu.oauth2.client.OAuthClient.accessToken(OAuthClient.java:55) at org.apache.oltu.oauth2.client.OAuthClient.accessToken(OAuthClient.java:71) at org.wso2.carbon.identity.application.authenticator.oidc.OpenIDConnectAuthenticator.getOauthResponse(OpenIDConnectAuthenticator.java:580) ... 61 more

    After this error, with the GLUU administration UI, I changed the "authentication method for token endpoint" from client_secret_basic to client_secret_post, and I tried the test again, and all works fine:

    1. Calling wso2is authorization endpoint from my "simple web app."
    2. Redirect to GLUU login page
    3. Redirect to WSO2IS auth page (OpenID User Claims Service Provider Test application requests access to your profile information ...)
    4. Back to my "simple web app" callback uri with the authorization code: {code=7269a75f-8702-3843-a553-465b69546a27, state=6yFD89, session_state=61d504abd9fe3cda28934f4aa394b156d6792071669f3020d4b79373ba5a1236.-QhpECv8pyocY1T2uu-kBQ}
    5. My "simple web app" calls the wso2is authorization endpoint, passing the code, and obtain the accecc_token

    And finally my question:

    In this test scenario I can change the configurations of the external IDP, but in a real-world scenario, I could not change that parameter on the external IDP. I could not find in the WSO2IS Identity Provider configuration how to choose from client_secret_basic, client_secret_post, client_secret_jwt, private_key_jwt, none.

    Without changing the GLUU IDP setting from client_secret_basic to client_secret_post what should I change in the wso2is's configuration to make this scenario work?

  • How Jhipster Gateway user can get the entities from a Service for the logged-in user ONLY.

    I'm using jhipster microservices to build my project called Sport Stat application. I have used jhipster to generate the following spring boot backend only microservices:

    • PlayerStatService
    • LeagueService

    The PlayerStatService has several entities such as Sport, Skill, Player and so on. Each player can have a list of Sessions, Targets and etc.

    Sports and Skills are all predefined.

    I also have generated a gateway client that communicates with the above services securely. You can sign up and register yourself as a User on the Gateway (client). A user can a be Coach who is monitoring his players. A coach can add a new player(s) and start monitoring and tracking their stats. A coach can have many players (one-to-many).

    Logged-in users (Coaches) should view their players and their stats only, NOT everybody else's. I assume I can have entity relationship between Player and User so that I can use findByUserIsCurrentUser to get the current user players and stats.

    I know this can be used easily if it a monolithic application.

    Problem: Because the User is generated by jhipster on gateway and Player is on PlayerStatService, I cannot create any relationship between them. I get this error.

    Your entity cannot have a relationship with User because it's a gateway entity

    My Question: Is it possible to achieve such a thing by using the generated classes and objects (Service, Repository)?

    If not, how would you suggest, for example: to restrict Users to be able to get their own players ONLY? When calling the PlayerStatService RestAPI to get players how do you suggest I should incorporate the logged-in user in my query at the repository level?

  • Spring Cloud Netflix Eureka Server side registry filtering

    I've been investigating the feasibility of introducing a filter / interceptor to manage some custom routing rules in a Spring Cloud Eureka Server. This is rather than having the applicable filtering duplicated across clients.

    I.e. The client service will register with some additional service metadata (eg. Major/Minor version) Requests to the server for that service will be made with additional consumer meta data (e.g. consumer identifier) The server side filter will check this data and remove entries from the registry response based on some filtering rules stored in the remote config (e.g. certain consumers prefer certain versions)

    Does anyone have any experience with performing something along these lines, as most of my attempts have stumbled (e.g. issues with the Netflix core gzip compression filter)

    I'm also concerned that I'm looking at this in the wrong way, and there is a simpler solution to achieve the same result.

    Cheers,

  • Use ES6 'import' instead of 'require' in microservices framework for Node.js

    I am using moleculer microservices framework for Node.js. Now, I need to use ES6 import instead of require.

    For Example, 

    const { abc } = require("../../index.js"); **// This is working well.** <br>
import { abc } from "../../index.js";      **// This isn't working and throw the error.**

    I am getting error like Unexpected token import. I've installed Babel(JavaScript compiler) as well.

    Do you have any idea about this?

    Thanks,

  • Kubernetes. Ajax API baseUrl resolution for Dynamic environment setup

    Gotta point out beforehand that I'm fairly new to K8S.

    So, the problem I'm trying to solve sounds like:

    I have 2 deployments(I have more but other ones don't really matter). One for front-end and one for backend API with following YAMLs(only important parts):

    kind: Deployment
spec:
  containers:
    - name: my-api
      image: my-api-image:latest
      ports:
        - containerPort: 80

---
kind: Service
spec:
  type: NodePort (could be LoadBalancer, doesn't matter for the case)
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
      name: http
  selector:
    app: my-api

    and

    kind: Deployment
spec:
    spec:
      containers:
      - name: my-ui
        image: my-ui-image:latest
        ports:
          - containerPort: 80
---
kind: Service
spec:
  type: NodePort
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
      name: http
  selector:
    app: my-ui

    So using this and kubectl apply, I can successfully spin 2 services up with no presetup whatsoever, I can also make calls from UI to API (inside the container) using kube-dns no prob. I can also spin 2 environments just by supplying --namespace argument or even by changing the cloud engine.

    The problem arrives when my UI needs to make an ajax request(from browser) to my API. Obviously, browser runs outside of container and cannot make any use of kube-dns and service name resolution.

    I've tried couple of solutions which were:

    • Proxying ajax requests through UI host to backend. Downside: Both service are too tied which affects independent scaling.
    • Ingress Controller set up in the way of: / points to my UI and /api points to my API so it's enough to use relative baseURL path on my UI. Downside: Really hard to spin new nginx-controller in another namespace(needing to create a user and a role just to spin it up), the solution overall seems too complicated just to spin a "feature environment".
    • I could modify existing controller to now serve a new 2 paths named like /{feature-name} for UI and /{feature-name} for an API but it forces me to edit ingress back if I wan't to kill my environment instead of just removing the namespace.

    Honestly I don't mind the problems that come with Ingress/API Gateway approaches, I'm just interested if I'm missing some piece.

    So the question goes: What's the best practice for external service URL resolution in Kubernetes?

  • How to enable ECS service discovery by ecs-cli?

    I want to use the functionality of ECS Service Discovery. https://aws.amazon.com/jp/blogs/news/amazon-ecs-service-discovery/

    I use ecs-cli for lanunching ECS fargate. However, I read the document of ecs-cli and cannot find any information.

    Does anyone know about it? Thanks in advance.

    https://github.com/aws/amazon-ecs-cli

  • Feign.RetryableException: Read timed out executing POST [URL]

    My feign call is getting read timeout, even after adding below property to my application.properties.

    hystrix.command.default.execution.timeout.enabled=false

    I have two more services, which got started successfully and got registered to my eureka.

    But my third (A) service is not starting due to readtime out issue. Below is the case

    A calls B (For authorization during application start up - Quartz Job)

    A calls C (flow ends) and application should start

    I have to start B & C before i start A. This is how it is been designed and i dont know the business case for this.

    After adding hystrix timeout, the service (A) is still throwing feign.RetryableException: Read timed out executing POST.

    I am not able to trouble shoot , since the request are routed through eureka services, and all services are running locally , there should not be any network latency.

    My eureka configuration

    spring.application.name=Foo-service eureka.client.registerWithEureka=true eureka.client.fetchRegistry=true eureka.client.serviceUrl.defaultZone=http://localhost:9094/eureka hystrix.command.default.execution.timeout.enabled=false