how to tell jackson serializer to stop continuing to serialize and write when object is big enough

2020-11-25 07:08 seiyas imported from Stackoverflow

in my recent project, there are a lot of log statement like below:

Bar bar = foo();
logger.info("something happened: param:{}", JSON.toJSONString(bar));

as you can see, just json serialize the object into log. sometimes bar is very big, and serializing the bar object consumes too much time, and log file expands a lot. the json serializing framework is fasterxml-jackson

So my question is: is there a way ,like customizing JsonSerializer to implement:

count the chars/bytes that has already been serialized(by invoking method #toJSONString())
if the count has exceed the MAX_SIZE, say 2000, then stop
return just the 2000 char String and print it in log file

2 answers

answered 2020-11-25 10:49 Michał Ziober

If we do not want to fall into Jackson and how it works we can just limit Writer where objects are serialised into. It is simple approach which does not require to implement custom serialisers with limits checked inside. Downside of this approach is we do not skip serialisation process we just ignore it's result.

You can write your own Writer with a maximum buffer size. Simple implementation based on StringBuilder:

class LimitedStringBuilderWriter extends Writer {

    private final StringBuilder buffer;
    private int remaining;

    public LimitedStringBuilderWriter(int limit) {
        if (limit <= 0) {
            throw new IllegalArgumentException("Limit must be positive number!");
        }

        this.remaining = limit;
        this.buffer = new StringBuilder(limit);
    }

    @Override
    public void write(char[] cbuf, int off, int len) {
        if (len == 0 || this.remaining <= 0) {
            return;
        }
        if ((off < 0) || (off > cbuf.length) || (len < 0) ||
                ((off + len) > cbuf.length) || ((off + len) < 0)) {
            throw new IndexOutOfBoundsException();
        }
        final int size = len - off;
        if (this.remaining >= size) {
            this.remaining -= size;
            buffer.append(cbuf, off, len);
            return;
        }
        buffer.append(cbuf, off, this.remaining);
        this.remaining = 0;
    }

    @Override
    public void write(int c) {
        if (this.remaining <= 0) {
            return;
        }
        this.remaining--;
        this.buffer.append(c);
    }

    @Override
    public void flush() {
    }

    @Override
    public void close() {
    }

    @Override
    public String toString() {
        return this.buffer.toString();
    }
}

Usage:

import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.json.JsonMapper;

import java.io.IOException;
import java.io.Writer;
import java.util.Arrays;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.function.Supplier;

public class LimitJsonLogApp {

    public static void main(String[] args) {
        Map<String, Object> value = new LinkedHashMap<>();
        value.put("array", Arrays.asList(1, 2, 3));
        value.put("string", "Value");
        value.put("int", 23);

        for (int limit = 11; limit < 30; limit += 3) {
            System.out.println(limit + " => " + JSON.toJSONString(value, limit));
        }
    }
}

class JSON {

    private static final ObjectMapper mapper = JsonMapper.builder().build();

    public static String toJSONString(Object value) {
        return toJSONStringSupplier(value).get();
    }

    public static String toJSONString(Object value, int limit) {
        return toJSONStringSupplier(value, limit).get();
    }

    public static Supplier<String> toJSONStringSupplier(Object value) {
        return toJSONStringSupplier(value, 1000);
    }

    public static Supplier<String> toJSONStringSupplier(Object value, int limit) {
        if (value == null) {
            return () -> "null";
        }
        return () -> {
            try {
                LimitedStringBuilderWriter writer = new LimitedStringBuilderWriter(limit);
                mapper.writeValue(writer, value);
                return writer.toString();
            } catch (IOException e) {
                throw new IllegalArgumentException(e);
            }
        };
    }
}

Above code prints:

11 => {"array":[1
14 => {"array":[1,2,
17 => {"array":[1,2,3],
20 => {"array":[1,2,3],"st
23 => {"array":[1,2,3],"strin
26 => {"array":[1,2,3],"string":
29 => {"array":[1,2,3],"string":"Va

Notice, some methods in JSON class returns Supplier<String>. You should use them if your logger allows to provide suppliers. It allows to postpone serialisation to the moment when it is really needed. And in case you disable INFO log in configuration it would really disable execution of this code.

answered 2020-11-25 14:43 seiyas

after doing some research, the below thought comes up. this is not that elegant. wish someone could improve this!

import java.io.IOException;
import java.io.Writer;
import java.util.stream.IntStream;

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.json.WriterBasedJsonGenerator;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.MappingJsonFactory;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationConfig;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.jsontype.TypeSerializer;
import com.fasterxml.jackson.databind.ser.BeanSerializerModifier;
import com.fasterxml.jackson.databind.ser.SerializerFactory;
import com.fasterxml.jackson.databind.ser.Serializers;
import org.jetbrains.annotations.NotNull;

public class JSON {

    static int MAX_SIZE = 10;
    final static ObjectMapper OBJECT_MAPPER;
    static {
        OBJECT_MAPPER = new ObjectMapper(new WrappedJsonFactory(new MappingJsonFactory()), null, null);
        OBJECT_MAPPER.setSerializerFactory(new WrapperSerialFactory(OBJECT_MAPPER.getSerializerFactory()));
    }

    static String toJSONString(Object value) {
        try {
            return OBJECT_MAPPER.writeValueAsString(value);
        } catch (IOException ex) {
            throw new RuntimeException(ex);
        }
    }


    public static void main(String... args) throws Exception {
        int[] array = IntStream.range(1, 100).toArray();
        System.out.println(JSON.toJSONString(array));
        MAX_SIZE = 20;
        System.out.println(JSON.toJSONString(array));
        MAX_SIZE = 30;
        System.out.println(JSON.toJSONString(array));
    }

    static class WrapperSerialFactory extends SerializerFactory {

        private SerializerFactory wrapped;

        public WrapperSerialFactory(SerializerFactory wrapped) {
            this.wrapped = wrapped;
        }

        @Override
        public SerializerFactory withAdditionalSerializers(Serializers additional) {
            return wrapped.withAdditionalSerializers(additional);
        }

        @Override
        public SerializerFactory withAdditionalKeySerializers(Serializers additional) {
            return wrapped.withAdditionalKeySerializers(additional);
        }

        @Override
        public SerializerFactory withSerializerModifier(BeanSerializerModifier modifier) {
            return wrapped.withSerializerModifier(modifier);
        }

        @Override
        public JsonSerializer<Object> createSerializer(SerializerProvider prov, JavaType baseType)
            throws JsonMappingException {
            JsonSerializer<Object> serializer = wrapped.createSerializer(prov, baseType);
            return new WrappedTypeSerializer(serializer);
        }

        @Override
        public TypeSerializer createTypeSerializer(SerializationConfig config, JavaType baseType)
            throws JsonMappingException {
            return wrapped.createTypeSerializer(config, baseType);
        }

        @Override
        public JsonSerializer<Object> createKeySerializer(SerializationConfig config, JavaType type,
            JsonSerializer<Object> defaultImpl) throws JsonMappingException {
            return wrapped.createKeySerializer(config, type, defaultImpl);
        }
    }



    static class WrappedJsonFactory extends JsonFactory {
        private JsonFactory wrapped;

        public WrappedJsonFactory(JsonFactory wrapped) {
            this.wrapped = wrapped;
        }

        @Override
        public JsonGenerator createGenerator(Writer w) throws IOException {
            w = new LimitingWriter(w);
            return wrapped.createGenerator(w);
        }
    }

    static class LimitingWriter extends Writer {
        private int count;
        private boolean finished = false;

        private Writer delegate;

        public LimitingWriter(Writer delegate) {
            this.delegate = delegate;
        }

        private void finish() throws IOException {
            if (!finished) {
                delegate.write("...");
                finished = true;
            }
        }

        @Override
        public void write(int c) throws IOException {
            if (!finished) {
                if (count >= MAX_SIZE) {
                    finish();
                } else {
                    delegate.write(c);
                    ++count;
                }
            }
        }

        @Override
        public void write(@NotNull char[] cbuf) throws IOException {
            if (!finished) {
                if ((count + cbuf.length) > MAX_SIZE) {
                    delegate.write(cbuf, 0, MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    delegate.write(cbuf);
                    count += cbuf.length;
                }
            }
        }

        @Override
        public void write(@NotNull String str) throws IOException {
            if (!finished) {
                if (count + str.length() > MAX_SIZE) {
                    delegate.write(str, 0, MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    delegate.write(str);
                    count += str.length();
                }
            }
        }

        @Override
        public void write(@NotNull String str, int off, int len) throws IOException {
            if (!finished) {
                if (count + len > MAX_SIZE) {
                    delegate.write(str, 0, MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    delegate.write(str, off, len);
                    count += len;
                }
            }
        }

        @Override
        public Writer append(CharSequence csq) throws IOException {
            if (!finished) {
                if (count + csq.length() > MAX_SIZE) {
                    count = MAX_SIZE;
                    delegate.append(csq, 0, MAX_SIZE - count);
                    finish();
                } else {
                    count += csq.length();
                    delegate.append(csq);
                }
            }
            return this;
        }

        @Override
        public Writer append(CharSequence csq, int start, int end) throws IOException {
            if (!finished) {
                if (count + end - start > MAX_SIZE) {
                    delegate.append(csq, start, start + MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    count += (end - start);
                    delegate.append(csq, start, end);
                }
            }
            return this;
        }

        @Override
        public Writer append(char c) throws IOException {
            if (!finished) {
                if (count >= MAX_SIZE) {
                    finish();
                } else {
                    ++count;
                    delegate.append(c);
                }
            }
            return this;
        }

        @Override
        public void write(@NotNull char[] cbuf, int off, int len) throws IOException {
            if (!finished) {
                if (count + len > MAX_SIZE) {
                    delegate.write(cbuf, off, MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    delegate.write(cbuf, off, len);
                    count += len;
                }
            }
        }

        @Override
        public void flush() throws IOException {
            delegate.flush();
        }

        @Override
        public void close() throws IOException {
            delegate.close();
        }
    }

    static class WrappedTypeSerializer extends JsonSerializer {
        private JsonSerializer wrapped;

        public WrappedTypeSerializer(JsonSerializer wrapped) {
            this.wrapped = wrapped;
        }

        @Override
        public void serialize(Object value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
            WriterBasedJsonGenerator writerBasedJsonGenerator = (WriterBasedJsonGenerator)gen;
            LimitingWriter writer = (LimitingWriter)writerBasedJsonGenerator.getOutputTarget();
            if (writer.finished) {
                return;
            }
            wrapped.serialize(value, gen, serializers);
        }
    }
}

output:

[1,2,3,4,5...
[1,2,3,4,5,6,7,8,9,1...
[1,2,3,4,5,6,7,8,9,10,11,12,13...

See also questions close to this topic

How to make Navigation drawer like this?

I want to make text like this with same space please help me friends
URL resource = this.getClass().getResource("file-name") is null
Completely puzzled. I do:
```
URL resource = this.getClass().getResource("eye-visible.png");
```
and the URL evaluates to null. "eye-visible.png" is a file in the same package as my class, which is "Controller.java":

I also tried (actually I started with this):
```
InputStream inputStream = Controller.class.getResourceAsStream("eye-visible.png");
```
and this returns null as well.

Really stuck - any clue??

This is a modular project and I run on the JDK14. I use NetBeans for an IDE. The lines above are in the start method of a JavaFX application. I reproduced the same null output with the lines above, in a new java project that has just a main class: same effect.
Enums Switch Statement for Type
```
public enum Black {

   BLACK,WHITE;
   private boolean BLACK() {
       boolean b = true;
       switch(this) {
           case BLACK: return true; 
           case WHITE: return false;
       }        
       return b;
   }        
}
```
For the above code, I am trying to check if my enum is "BLACK", or "WHITE". However, I tried debugging this and when I have an instance of "BLACK", in the switch statements, both cases evaluate to true. What am I doing wrong and how can I fix this?

How to set the log4j output directory to a directory from a java .properties file

I have a program that reads in a .properties file, and then sends outputs to whatever path is defined in that .properties file.

String userDirectory = System.getProperty("user.dir");
// read properties input
Properties userProperties = new Properties();
FileInputStream in;
try {
    in = new FileInputStream("somePath");
    userProperties.load(in);
    in.close();
} catch (Exception e) {
    e.printStackTrace();
}

// Set userProperties
String outputs = userProperties.getProperty("run.outputs");
String logs = userProperties.getProperty("run.logs");

In my program, each class has a logger object defined at the top like so:

public class Car
{
    static Logger log = Logger.getLogger(Car.class);      // Logging
.....
}

How can I set all the loggers in the program to be written to whatever directory I defined in my .properties file under String logs = userProperties.getProperty("run.logs");?

Configure promtail 2.0 to read the files .log

Since I've updated to promtail 2.0, I'm unable to read the content of a log file in loki.

config-promtail.yml

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://192.168.1.103:3100/loki/api/v1/push

scrape_configs:
  - job_name: manuallog
    static_configs:
      - targets:
          - 192.168.1.103
        labels:
          job: tomcat
          host: 192.168.1.103
          path: /opt/error.log

I've also tried to use a different configuration in the scrape config, but with no luck:

  - job_name: varlog
journal:
  max_age: 12h
  labels:
    filename: /opt/error.log
    path: /opt/error.log

The error.log is not empty:

    # cat /opt/error.log
Disconnected from localhost

The Promtail version - 2.0

    ./promtail-linux-amd64 --version
promtail, version 2.0.0 (branch: HEAD, revision: 6978ee5d)
  build user:       root@2645337e4e98
  build date:       2020-10-26T15:54:56Z
  go version:       go1.14.2
  platform:         linux/amd64

Any clue? Am I doing anything wrong?

Many thanks,

Log4j2 Async Logging RollingFile JsonLayout

I want to include in my project a rolling logging file but with json inside instead of typical PatternLayout. I succeed and I wanted to make it asynchronous to have better performances but nothing is logging in my file as soon as I change my Logger to AsyncLogger. I don't know if I have missed something in the documentation about incompatibility between AsyncLogging and JsonLayout Rolling File. So here I am, asking you for help !

Find below my log4j2.xml configuration for my project :

<Configuration>
    <Properties>
        <Property name="LOG_PATTERN">
            %d{yyyy-MM-dd HH:mm:ss} [%t] %-5level %c{1.} - %msg - [${sys:hostName}] %n
        </Property>
    </Properties>
    <Appenders>
        <!-- Rolling File Appender -->
        <RollingFile name="File" fileName="@log4jFileValue@/report-business.log"
            filePattern="@log4jFileValue@/report-business.log-%d{yyyy-MM-dd}-%i.gz">
            <PatternLayout>
                <Pattern>${LOG_PATTERN}</Pattern>
            </PatternLayout>
            <Policies>
                <!-- To rollover logs on daily basis, set interval to 1 in TimeBasedTriggeringPolicy -->
                <TimeBasedTriggeringPolicy interval="1" />
                <!-- To rollover log files based on log file size -->
                <SizeBasedTriggeringPolicy size="100MB" />
            </Policies>
            <!-- Historical logs day save -->
            <DefaultRolloverStrategy max="90" />
        </RollingFile>

        <!-- Rolling File Appender -->
        <RollingFile name="JsonFile" fileName="@log4jFileValue@/report-json.log"
            filePattern="@log4jFileValue@/report-json.log-%d{yyyy-MM-dd}-%i.gz">
            <JSONLayout  compact="true"
                         complete="false"
                         objectMessageAsJsonObject="true"
                         includeStacktrace="false"
                         includeNullDelimiter="false"
                         eventEol="true">
                <KeyValuePair key="kind" value="DLR"/>
            </JSONLayout>
            <Policies>
                <!-- To rollover logs on daily basis, set interval to 1 in TimeBasedTriggeringPolicy -->
                <TimeBasedTriggeringPolicy interval="1" />
                <!-- To rollover log files based on log file size -->
                <SizeBasedTriggeringPolicy size="100MB" />
            </Policies>
            <!-- Historical logs day save -->
            <DefaultRolloverStrategy max="90" />
        </RollingFile>
    </Appenders>
    <Loggers>
        <AsyncLogger name="LOGGER_JSON" additivity="false" level="info" includeLocation="true">
            <AppenderRef ref="JsonFile"/>
        </AsyncLogger>
        <Root level="@logLevel@" includeLocation="true">
            <AppenderRef ref="File" />
            <AppenderRef ref="MailAppender" />
        </Root>
    </Loggers>
</Configuration>

Just a quick note that "@log4jFileValue@" is a variable depending of build profile defined in filters files.

Thanks in advance for your help

PS : Sorry for my awful english level from France !

Modify property serialization/deserialization in WinForms Form1.resx file

I have a custom class and made the same serialization as System.Drawing.Image, by implementing ISerializable interface and two methods:

void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context) to give the serializer my data in the format I need.
A constructor in my class: protected MyClass(SerializationInfo info, StreamingContext context)


    [Serializable]
    [TypeConverter(typeof(MyClassTypeConverter))]
    [Editor(typeof(MyClassEditor), typeof(UITypeEditor))]
    public class MyClass : MarshalByRefObject, ISerializable, ICloneable, IDisposable

A have added my class as a property to a custom button -> MyButton.MyClass The problem is that at design time when I set MyClass the serializer writes the assembly name and version in the RESX file of the form before my custom serialized object. Here is how it looks like:


    <assembly alias="MyAssembly" name="MyAssembly, Version=2020.1.1, Culture=neutral, PublicKeyToken=XXX" />
      <data name="myButton1.MyClass" type="MyAssembly.MyClass, MyAssembly">
    <value>Some long text data</data>
      </data>

After a new release my clients started having this exception:


    System.InvalidCastException: '[A]MyAssembly.MyClass cannot be cast to [B]MyAssembly.MyClass. Type A originates from 'MyAssembly, Version=2020.1.1, Culture=neutral, PublicKeyToken=XXX' in the context 'Default' at location 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\...

Here is the designer file:


    private void InitializeComponent()
    {
        System.ComponentModel.ComponentResourceManager resources = new System.ComponentModel.ComponentResourceManager(typeof(Form2));
        this.myButton1 = new MyAssembly.MyButton();
        this.SuspendLayout();
        // 
        // myButton1
        // 
        this.myButton1.Location = new System.Drawing.Point(5, 5);
        this.myButton1.Name = "myButton1";
        this.myButton1.Size = new System.Drawing.Size(200, 25);
        this.myButton1.MyClass = ((MyAssembly.MyClass)(resources.GetObject("myButton1.MyClass")));
        this.myButton1.TabIndex = 0;
        this.myButton1.Text = "myButton1";

The assembly where type MyClass is located is already loaded, because MyButton is located in the same assembly. So I do not need the RESX file to contain information about the assembly. Currently I cannot find a solution on my own and I am thinking of two options:

How can I prevent serializing the <assembly alias="MyAssembly"...> in the resx file of the form?
Can I load MyClass from the resx file without loading the described assembly?

Function as a parameter in celery task

I have an app, where I want to use multiprocessing with celery. I get errors every time I try to run it, so now I try to make a simple version just to reach normal process of working.

celery_module/my_celery.py:

from celery import Celery

app1 = Celery('cel_tasks',
              broker='redis://localhost:6379/0',
              backend='redis://localhost',
              include=['celery_module.tasks.my_tasks'],
              )

app1.conf.update(
    task_serializer='pickle',
    accept_content=['pickle', 'json']
)

celery_module/tasks/my_tasks.py:

from celery_module.my_celery import *

@app1.task
def do_simple(func):
    return func()*10

src/celery_launch.py:

import celery_module.tasks.my_tasks
from src.func_serialization import deserialize_function

    def start_simple(des_func):
        des_func = deserialize_function(des_func)
        new_func = celery_module.tasks.my_tasks.do_simple.delay(des_func)
        return new_func

src/func_serialization.py:

import marshal
import types


def serialize_function(function):
    serialized_func = marshal.dumps(function.__code__)
    return serialized_func


def deserialize_function(function_obj):
    func_repr = marshal.loads(function_obj)
    deserialized_function = types.FunctionType(func_repr, globals())
    return deserialized_function

src/app.py:

from src.celery_launch import start_simple
from src.func_serialization import serialize_function

@staticmethod
def simple_func():
    x = 5
    return x*10

def some_function(self):  #here is some logic of my app basically, but now here is only test function, just because it's easier to call the func from UI
    start_simple(serialize_function(self.simple_func))

So I call the some_function from app.py. As I pass a function in a start_simple(), I serialize it. Then start_simple() deserializes my simple_func() and sends it as a parameter to a celery task do_simple.

So the error I get is: kombu.exceptions.EncodeError: Can't pickle <function simple_func at 0x7f3066806d30>: attribute lookup simple_func on src.func_serialization failed

How could I fix it?

Using RestTemplate to retrieve and deserialize JSON of complex object

I'm consuming a REST API using RestTemplate like so

public CoinResponse getExchange(){
        List<HttpMessageConverter<?>> messageConverters = new ArrayList<>();
        MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
        converter.setSupportedMediaTypes(Collections.singletonList(MediaType.ALL));
        messageConverters.add(converter);
        restTemplate.setMessageConverters(messageConverters);
       return restTemplate.getForObject("https://api.coindesk.com/v1/bpi/currentprice.json", CoinResponse.class);
    }

And I'm getting the response as an instance of CoinResponse. Within CoinResponse I can access the BpiObject instance but the EUR, USD and GBP instances are all null. My model looks like this:

public class CoinResponse {
    public CoinResponse() {
    }

    @JsonProperty
    Time TimeObject;
    @JsonProperty
    private String disclaimer;
    @JsonProperty
    private String chartName;
    @JsonProperty
    Bpi BpiObject;
    // Getters and Setters
}

public class Bpi {
    public Bpi() {
    }

    @JsonProperty
    USD USDObject;
    @JsonProperty
    GBP GBPObject;
    @JsonProperty
    EUR EURObject;
    // Getter and Setters

What am I doing wrong here? What's the correct way of addressing this issue with RestTemplate?

My currency class looks like this:

public class EUR {
    public EUR() {
    }

    @JsonProperty
    private String code;
    @JsonProperty
    private String symbol;
    @JsonProperty
    private String rate;
    @JsonProperty
    private String description;
    @JsonProperty
    private float rate_float;
    // Getter and Setters
}

Serializing multiline string from JsonNode to YAML string adds double quotes and "\n"
I have a YAML string where one of the attributes looks like this:
```
 description: |
    this is my description
    this is my description in the second line
```
In my Java code I read it into a JsonNode like this:
```
JsonNode node = new YamlMapper().readTree(yamlString);
```
I then do some changes to it and write it back to a string like this:
```
new YamlMapper().writeValueAsString(node))
```
The new string now looks like this:
```
"this is my description\nthis is my description in the second line\n"
```
So now in the YAML file you can see the added quotes + the new line character (\n) and everything is in one line.

This is how my YAML object mapper is configured:
```
 new ObjectMapper(
        new YAMLFactory()
          .disable(YAMLGenerator.Feature.MINIMIZE_QUOTES))
        .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
        .setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
```
What can I do to not have quotes added around my string when serializing the JsonNode to a String?
Sending Json along with multipart file in SpringBoot
I'm trying to figure out how to pass a collection of optional parameters to a SpringBoot controller, along with a multipart file, but can't figure out how.

Without the multipart file, this works fine
```
public ResponseEntity<ViewModel> performOperationOnFilePath(@Valid @RequestBody final FileOperationFileContract requestBody) 
```
where FileOperationFileContract is a POJO that looks like this
```
public class FileOperationFileContract {
    @NotBlank(message = "The filepath is required")
    private String filePath;

    private Map<String, String> options = new HashMap<>();

    @JsonAnyGetter
    public Map<String, String> getOptions() {
        return options;
    }

    @JsonAnySetter
    public void setOptions(final String name, final String value) {
        options.put(name, value);
    }
}
```
Here's what I tried with the multipart file
```
public ResponseEntity<ViewModel> performOperationOnFile(@RequestParam(name="myfile") final MultipartFile file, @RequestPart(name="options", required=false) final FileOperationMultipartFileContract requestBody)
```
where FileOperationMultipartFileContract is a POJO that looks like this
```
public class FileOperationMultipartFileContract {
    private Map<String, String> options = new HashMap<>();

    @JsonAnyGetter
    public Map<String, String> getOptions() {
        return options;
    }

    @JsonAnySetter
    public void setOptions(final String name, final String value) {
        options.put(name, value);
    }
}
```
This seems to work if I specify the Json payload in Postman like this (note that I must specify the content-type as application/json, or I get the message "'application/octet-stream' not supported"

But if there is no json part sent, I get a 403. I want the option collection to just be empty or null. In other words, I want it to default to {}

I've tried RequestPart and RequestParam, but there's no difference. I've also tried it with and without the @Valid annotation

How can I do this?

Update: I think that I figured out that my problem does not have as much to do with Spring as it does with what I'm trying to do with the @JsonAnyGetter and @JsonAnySetter annotations. If I change FileOperationFileContract to have regular fields, it works. But my guess is that the way I have it doesn't work very well with null. Although if the value is null, there should be no calls to setOptions. Any ideas?

Unable to deserialise via mixin

Attached is my json. I am receiving this from an endpoint. I have the object type in my consuming app. the object contains; success, message and loggedInMember (which is an object). In the json, the feeClasses contains a lot of fields and objects etc which is why i have not included the whole json. I am only interested in the success and message fields.

{
    "header":{
    "messageId":null,
            "receivedTimestamp":1611246394839,
            "replyTo":null,
            "redelivered":false,
            "originator":null
},
    "internalId":null,
        "auditPoints":[

],
        "success":true,
        "message":"",
        "loggedInMember":{
            "feeClasses":{
...CONTAINS A LOT OF FIELDS...
    }
}
}

I am trying to map this response to the object type, hence essentially leaving loggedInMembers null. This is my test I am trying to run:

   public void test() throws JsonProcessingException
   {
      //String json = "{\"header\":{\"messageId\":null,\"bucketNumber\":null,\"senderSubId\":null,\"senderLocationId\":null,\"onBehalfOfCompId\":null,\"onBehalfOfSubId\":null,\"onBehalfOfLocationId\":null,\"correlationId\":null,\"receivedTimestamp\":1611246394839,\"replyTo\":null,\"redelivered\":false,\"originator\":null},\"internalId\":null,\"auditPoints\":[],\"success\":true,\"message\":\"\",\"loggedInMember\":{\"memberLoginName\":\"BMARTINTEST\",\"memberId\":\"201901241246290000036402D\",\"settlementAccountIds\":[\"201901241246290000036491D\"],\"parentMemberId\":\"1\",\"firmId\":\"990\",\"memberType\":\"INDIVIDUAL\",\"memberAccountType\":\"PROD\",\"password\":\"D1208B304FD7AA6187690A389A5040C1D9B07643\",\"feeClasses\":{\"byId\":{\"201902120947520000559606D\":{\"memberLoginName\":\"BMARTINTEST\",\"feeClassId\":\"201508041827550000942152D\",\"memberFeeClassId\":\"201902120947520000559606D\",\"allocatedDate\":{\"year\":2019,\"month\":2,\"day\":12,\"timeMillis\":1549929600000},\"firstUsedForTradeDate\":{\"year\":2019,\"month\":2,\"day\":12,\"timeMillis\":1549929600000},\"firstUsedForSettlementDate\":null,\"usableFromDate\":{\"year\":2019,\"month\":2,\"day\":12,\"timeMillis\":1549929600000},\"usableToDate\":{\"year\":2019,\"month\":2,\"day\":19,\"timeMillis\":1550534400000},\"usableToTimestamp\":1550613600000,\"usableBusinessDaysAllocated\":6,\"usableBusinessDaysRemaining\":0,\"narrative\":\"Bonus assigned to member at first-time funding of amount 4000.00 : Set expiration date/time\",\"disabled\":false,\"usableForTrade\":true,\"usableForSettlement\":true},\"202001290940390000868824D\":{\"memberLoginName\":\"BMARTINTEST\",\"feeClassId\":\"202001290940340000776406D\",\"memberFeeClassId\":\"202001290940390000868824D\",\"allocatedDate\":{\"year\":2020,\"month\":1,\"day\":29,\"timeMillis\":1580256000000},\"firstUsedForTradeDate\":null,\"firstUsedForSettlementDate\":null,\"usableFromDate\":{\"year\":2020,\"month\":1,\"day\":6,\"timeMillis\":1578268800000},\"usableToDate\":{\"year\":2020,\"month\":2,\"day\":27,\"timeMillis\":1582761600000},\"usableToTimestamp\":1582840800000,\"usableBusinessDaysAllocated\":0,\"usableBusinessDaysRemaining\":0,\"narrative\":\"Added NO_FEES_CLASS\",\"disabled\":false,\"usableForTrade\":true,\"usableForSettlement\":true},\"201901241246290000036417D\":{\"memberLoginName\":\"BMARTINTEST\",\"feeClassId\":\"201508041736360000943781D\",\"memberFeeClassId\":\"201901241246290000036417D\",\"allocatedDate\":{\"year\":2019,\"month\":1,\"day\":24,\"timeMillis\":1548288000000},\"firstUsedForTradeDate\":null,\"firstUsedForSettlementDate\":null,\"usableFromDate\":{\"year\":2019,\"month\":1,\"day\":24,\"timeMillis\":1548288000000},\"usableToDate\":null,\"usableToTimestamp\":null,\"usableBusinessDaysAllocated\":0,\"usableBusinessDaysRemaining\":0,\"narrative\":null,\"disabled\":false,\"usableForTrade\":true,\"usableForSettlement\":true}},\"empty\":false},\"legalName\":\"Martin Birch\",\"taxId\":\"345335454\",\"taxCountryId\":\"US\",\"currency\":\"USD\",\"lastTradeId\":null,\"introducingBrokerMemberId\":null,\"introducingBrokerMemberName\":null,\"introducingBrokerMemberCode\":null,\"clearedByMemberId\":\"SECOND_TEST\",\"clearedByMemberLoginName\":null,\"memberProblems\":[],\"emailNotificationEnabled\":true,\"rtafLevelId\":0,\"rtafAmount\":0,\"maxNumberOfPositionAccounts\":1,\"ciciIdentifier\":null,\"traderRequired\":false,\"interestClass\":\"INDIVIDUAL\",\"memberCreatedDate\":1548333989000,\"parentMemberLoginNames\":[\"NADEX.COM\",\"NADEX\"],\"demoStartDate\":null,\"demoEndDate\":null,\"clientIdMaxLimit\":null,\"memberAccountApplicationFieldData\":null,\"rank\":0,\"uuid\":\"201901241246290000036395D\",\"referrerId\":\"raf4qam5h00s36d\",\"testMember\":false},\"allReplyToSource\":[],\"sendToOriginatorOnly\":false}";
String json = "{\n" +
              "   \"header\":{\n" +
              "      \"messageId\":null,\n" +
              "      \"receivedTimestamp\":1611246394839,\n" +
              "      \"replyTo\":null,\n" +
              "      \"redelivered\":false,\n" +
              "      \"originator\":null\n" +
              "   },\n" +
              "   \"internalId\":null,\n" +
              "   \"auditPoints\":[\n" +
              "      \n" +
              "   ],\n" +
              "   \"success\":true,\n" +
              "   \"message\":\"\",\n" +
              "   \"loggedInMember\":{\n" +
              "      \"feeClasses\":{\n" +
              "         \n" +
              "      }\n" +
              "   }\n" +
              "}";
      ObjectMapper objectMapper = new ObjectMapper();
      objectMapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
      objectMapper.addMixIn(LogonResponseMessage.class, LogonResponseMixin.class);

      LogonResponseMessage responseMessage = objectMapper.readValue(json, LogonResponseMessage.class);

      System.out.println(responseMessage);
   }

My mixin:

public abstract class LogonResponseMixin
{
   LogonResponseMixin(@JsonProperty("success") boolean success, @JsonProperty("message") String message){};

   @JsonIgnore
   abstract Member loggedInMember();

   @JsonIgnore
   abstract MemberFeeClasses feeClasses();

   @JsonIgnore
   abstract Header header();
}

I am getting the following error: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of domain.xyz.MemberFeeClasses (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)

Am i creating the mixin wrong? I have asked in a previous question and using mixin was the general consensus but it doesn't seem to play ball with me.

Thank you.

Jackson only serializes properties in the superclass
I'm using Jackson through retrofit with a very simple class hierarchy. The issue is that Jackson is only serializing the properties that are present in the superclass, but not any of the ones added in both subclasses.

An example:
```
abstract class A { int a; }

class B extends A { int b; }

class C extends A { int c; }
```
The output includes only a!

(I don't care about deserialization so I don't need to include type info or deal with polymorphic object creation)

Thanks!

What is the ideal way to serialize and deserialize polymorphic entity attribute in spring boot?

I have an Entity class with a column attribute whose type is an abstract class. I want to serialize (object to JSON string) while saving it in the database column and deserialize it into an abstract class (which in turn converts the string to the appropriate concrete class) when it is retrieved from the database.

Here's how I accomplished it:

ProductEntity.java

@Entity
@Table(name="PRODUCT")
@Data
public class ProductEntity{
    @Id
    @Column(name = "ID", insertable = false)
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private BigInteger id;

    @Column(name = "DESCRIPTION")
    private String description;

    @Column(name = "NAME")
    private String name;

    @Column(name = "PRODUCT_TYPE")
    private String productType; 

    @Column(name = "PRODUCT_SPECS")
    @JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.EXTERNAL_PROPERTY, property = 
    "productType") // -------------------> Map to concrete class based on productType value
    @Convert(converter = ObjectConverter.class) // ------------> custom converter
    private ProductSpecification productSpec;
}

NOTE : "PRODUCT_SPECS" database column is of JSON type.

ProductSpecification.java

@NoArgsConstructor
@JsonTypeInfo(use = JsonTypeInfo.Id.MINIMAL_CLASS,
    include = JsonTypeInfo.As.WRAPPER_OBJECT,
@JsonSubTypes({
    @JsonSubTypes.Type(value = ComputerSpecification.class, name = "computer"),
    @JsonSubTypes.Type(value = SpeakerSpecification.class, name = "speaker")
})
public abstract class ProductSpecification{ }

ComputerSpecification.java

@Getter
@Setter
@NoArgsConstructor
@JsonTypeName("computer")
public class ComputerSpecification extends ProductSpecification {
String memory;
String displaySize;
String processor;

@JsonCreator
public ComputerSpecification (@JsonProperty("memory") String memory, 
                              @JsonProperty("displaysize") String displaySize,
                              @JsonProperty("processor") String processor){
    super();
    this.memory = memory;
    this.displaySize = displaySize;
    this.processor = processor;
   }
}

SpeakerSpecification.java

@Getter
@Setter
@NoArgsConstructor
@JsonTypeName("computer")
public class SpeakerSpecification extends ProductSpecification {
String dimension;
String sensitivity;
String bassPrinciple;
String amplifierPower;

@JsonCreator
public SpeakerSpecification (@JsonProperty("sensitivity") String sensitivity, 
                              @JsonProperty("dimension") String dimension,
                              @JsonProperty("bassPrinciple") String bassPrinciple,
                              @JsonProperty("amplifierPower") String amplifierPower){
    super();
    this.sensitivity = sensitivity;
    this.dimension = dimension;
    this.bassPrinciple = bassPrinciple;
    this.amplifierPower = amplifierPower;
   }
}

ObjectConverter.java

NOTE: I am using Jackson ObjectMapper for serialization and deserialization.

public class ObjectConverter implements AttributeConverter<Object, String>{
private final static Logger LOGGER = LoggerFactory.getLogger(ObjectConverter.class);
private static final ObjectMapper mapper;

static {
    mapper = new ObjectMapper();
    mapper.setSerializationInclusion(Include.NON_NULL);
}

@Override
public String convertToDatabaseColumn(Object attributeObject) {
    if (attributeObject == null) {
        return "";
    }
    try {
        return mapper.writeValueAsString(attributeObject);
    } catch (JsonProcessingException e) {
        LOGGER.error("Could not convert to database column", e);
        return null;
    }
}

@Override
public Object convertToEntityAttribute(String dbColumnValue) {
    try {
        if (StringUtils.isBlank(dbColumnValue)) {
            return null;
        }
        return mapper.readValue(dbColumnValue, ProductSpecification.class); // ----> mapped to 
                                                                                    abstract class
    } catch (Exception e) {
        LOGGER.error("Could not convert to entity attribute", e);
        return null;
        }
    }
}

Request body 1:

{
    "name" : "Bose Bass Module 700 - Black- Wireless, Compact Subwoofer",
    "description" : "This wireless, compact subwoofer is designed to be paired with the Bose sound 
                     bar 700 to bring music, movies, and TV to life with Deep, dramatic bass. ",
    "productSpec" : {
                       "sensitivity" : "90 dB",
                       "bassPrinciple" : "reflex",
                       "amplifierPower" : "700 watts",
                       "dimension" : "14-5/16inW x 42-13/16inH x 16-5/16inD"
                    }
}

This request gets saved in the database column "PRODUCT_SPECS" as :

{".SpeakerSpecification ":{"sensitivity" : "90 dB","bassPrinciple" : "reflex", "amplifierPower" :"700 
watts", "dimension" : "14-5/16inW x 42-13/16inH x 16-5/16inD" }}

Now this solution works perfectly fine. The "SpeakerSpecification " key neither appears in the response of GET API call nor in the swagger doc. But having to store the type info in the database really bothers me.

Is there a better approach to this problem where I could avoid having the typeinfo (".SpeakerSpecification ") in the column value?

Getting Exception in thread "main" java.lang.NoSuchFieldError: ACCEPT_CASE_INSENSITIVE_ENUMS
I have the following code
```
private val objectMapper: ObjectMapper = new ObjectMapper().registerModule(new DefaultScalaModule)
     .registerModule(new JavaTimeModule)
     .configure(FAIL_ON_UNKNOWN_PROPERTIES, false)
.enable(MapperFeature. ACCEPT_CASE_INSENSITIVE_ENUMS)
```
Its this enable thats throwing that error. I checked all the versions of Jackson-Module and there are no conflicts, its correct version. How do i figure out and fix this. This looks like Dependency but i cannot seem to figure this out. Everything looks correct to me. I looked at the Dependency tree as well.

Related questions

how to tell jackson serializer to stop continuing to serialize and write when object is big enough

2 answers

See also questions close to this topic