Stream using Comparator dropping values

I am a junior developer with little experience using Java streams. I have identified (through some extra logging) the following stream is dropping a record that I am specifically looking for but I can't figure out why. Please note, I did not write this code and the person that did has left the company. This project is using Spring.

ConfirmCode table is structured as follows:

  Name              Type    Length  Not Null
  confirm_code_id   int     11      True
  account_id        varchar 20      False
  product_id        varchar 20      False
  dimension         varchar 55      False
  confirm_code      int     1       False
  confirm_desc      varchar 50      False
  action_enum       enum    0       False 
  sms_keywords      varchar 255     False 

This table contains specific enums with null allowed for account_id, product_id and dimension by design:

  ConfirmCode
  accountId productId dimension code desc               enum     keywords
                                  2  Need to Reschedule RESCHED  {not important}
                                  7  Call Transferred   TRANSFER {not important}
                                 -1  Help               HELP     {not important}
                                 -2  Opt In             OPT IN   {not important}
                                 -3  Opt Out            OPT OUT  {not important}
                                 -4  Response           RESPONSE {not important}    
                                  1  Confirmed          CONFIRM  {not important}

However, a customer (accountID) can override the code that is tied to an enum by default. For example:

  ConfirmCode
  accountId productId dimension             code desc               enum     keywords
  704442    RemindMe  Appointment Reminder  2    Confirmed          CONFIRM  {not important}    

Since accountId, productId and dimension are allowed to be null, the query run by Spring needs to take that into account as well as a customer overriding the defaults as you can see above (parent account is null in this case).

Query:
public interface ConfirmCodeRepository extends JpaRepository<ConfirmCode, Long> {

  @Query("SELECT c FROM ConfirmCode c WHERE (accountId is null OR accountId = :parentAccountId OR accountId = :accountId) "
        + " AND (productId = :productId OR productId is NULL)"
        + " AND (dimension is null OR dimension = :dimension)")

  public List<ConfirmCode> findByAccountIdAndProductIdAndDimension(
        @Param("accountId") String accountId, @Param("parentAccountId") String parentAccountId,
        @Param("productId") String productId, @Param("dimension") String dimension);

The problem I'm having is that when the ConfirmCode table is queried for customer 704442, that record is returned for that customer but later dropped after going through the Comparator code in the stream. Only the records where accountId, productId and dimension are null. Here is the code which is doing the logic on the result set from the query:

@Repository
public class ConfirmCodeDAO {

    @Autowired
    ConfirmCodeRepository confirmCodeRepo;
    private static final Logger logger = LoggerFactory.getLogger(PreferenceService.class);

    public Set<ConfirmCode> findByAccountIdAndParentAccountIdAndProductIdAndDimension(
            String accountId,
            String parentAccountId, String productId, String dimension) {

        List<ConfirmCode> list = confirmCodeRepo
                .findByAccountIdAndProductIdAndDimension(accountId, parentAccountId, productId,
                        dimension);
        list.forEach(r -> logger.info("list: " + r.toString()));

        Set<ConfirmCode> set = confirmCodeRepo
                .findByAccountIdAndProductIdAndDimension(accountId, parentAccountId, productId,
                        dimension)
                .stream()
                .sorted(Comparator.comparing(ConfirmCode::getActionEnum,
                        Comparator.nullsFirst(Comparator.naturalOrder())))
                .sorted(Comparator.comparing(ConfirmCode::getDimension,
                        Comparator.nullsFirst(Comparator.naturalOrder())))
                .sorted(Comparator.comparing(ConfirmCode::getProductId,
                        Comparator.nullsFirst(Comparator.naturalOrder())))
                .sorted(Comparator.comparing(ConfirmCode::getAccountId,
                        Comparator.nullsFirst(Comparator.reverseOrder())))
                .collect(Collectors.toMap(ConfirmCode::getConfirmCode, confirmCode -> confirmCode,
                        (c1, c2) -> c2))
                .entrySet()
                .stream()
                .map(e -> e.getValue())
                .collect(Collectors.toSet());
        set.forEach(r -> logger.info("set2: " + r.toString()));
        return set;
    }
}

Here is the logging for the "list" object:

Calling confirmCodeDAO with: 704442, parentAccountId: null, productId: RemindMe, dimension: Appointment Reminder
list: [accountId=704442, productId=RemindMe, dimension=Appointment Reminder, confirmCode=2, confirmDesc=Confirmed, actionEnum=CONFIRM, smsKeywords={not important}]
list: [accountId=null, productId=null, dimension=null, confirmCode=1, confirmDesc=Confirmed, actionEnum=CONFIRM, smsKeywords={not important}]
list: [accountId=null, productId=null, dimension=null, confirmCode=2, confirmDesc=Need to Reschedule, actionEnum=RESCHED, smsKeywords={not important}]
list: [accountId=null, productId=null, dimension=null, confirmCode=7, confirmDesc=Call Transferred, actionEnum=TRANSFER, smsKeywords={not important}, account=null, responseType=null]
list: [accountId=null, productId=null, dimension=null, confirmCode=3, confirmDesc=Repeat, actionEnum=REPEAT, smsKeywords={not important}, account=null, responseType=null]
list: [accountId=null, productId=null, dimension=null, confirmCode=-1, confirmDesc=Help, actionEnum=HELP, smsKeywords={not important}, account=null, responseType=null]
list: [accountId=null, productId=null, dimension=null, confirmCode=-2, confirmDesc=Opt In, actionEnum=OPT IN, smsKeywords={not important}, account=null, responseType=null]
list: [accountId=null, productId=null, dimension=null, confirmCode=-3, confirmDesc=Opt Out, actionEnum=OPT OUT, smsKeywords={not important}, account=null, responseType=null]
list: [accountId=null, productId=null, dimension=null, confirmCode=-4, confirmDesc=Response, actionEnum=RESPONSE, smsKeywords={not important}, account=null, responseType=null]
list: [accountId=null, productId=null, dimension=null, confirmCode=-5, confirmDesc=Cancelled, actionEnum=CANCEL, smsKeywords={not important}, account=null, responseType=null]

Here is the logging for the "set" object:

set: [accountId=null, productId=null, dimension=null, confirmCode=-1, confirmDesc=Help, actionEnum=HELP, smsKeywords={not important}]
set: [accountId=null, productId=null, dimension=null, confirmCode=1, confirmDesc=Confirmed, actionEnum=CONFIRM, smsKeywords={not important}]
set: [accountId=null, productId=null, dimension=null, confirmCode=-5, confirmDesc=Cancelled, actionEnum=CANCEL, smsKeywords={not important}]
set: [accountId=null, productId=null, dimension=null, confirmCode=7, confirmDesc=Call Transferred, actionEnum=TRANSFER, smsKeywords={not important}]
set: [accountId=null, productId=null, dimension=null, confirmCode=3, confirmDesc=Repeat, actionEnum=REPEAT, smsKeywords={not important}, account=null, responseType=null]
set: [accountId=null, productId=null, dimension=null, confirmCode=-2, confirmDesc=Opt In, actionEnum=OPT IN, smsKeywords={not important}, account=null, responseType=null]
set: [accountId=null, productId=null, dimension=null, confirmCode=-4, confirmDesc=Response, actionEnum=RESPONSE, smsKeywords={not important}, account=null, responseType=null]
set: [accountId=null, productId=null, dimension=null, confirmCode=-3, confirmDesc=Opt Out, actionEnum=OPT OUT, smsKeywords={not important}, account=null, responseType=null]

You can see that after going through the stream code the record I need is dropped. Please advise on this and thanks for your time!

1 answer

  • answered 2018-11-18 21:14 cornz

    The sorted itself cannot drop elements. But by collecting the list using

    toMap.collect(Collectors.toMap(ConfirmCode::getConfirmCode, confirmCode -> 
    confirmCode, (c1, c2) -> c2))
    

    you only keep one for every property confirmCode.

    This toMap collector works as followes:

    The first parameter defines the key, so it takes the property confirmCode of the object ConfigmCode as key (names are a bit misleading). The second one is the value confirmCode -> confirmCode, so the whole object is kept as value. Hence we get a Map<Integer, ConfirmCode>.

    Third parameter describes how to merge two values if the key aready exists in the map. The functions (c1,c2) -> c2 simply says: take the second one.

    In your list the first and third element both have confirmCode=2, hence only the second one is kept in the map.

    As discussed in the comments this only drops the second element (in input list) with confirmItem=2 because of the comparators which sort null first.

    The map now contains the expected elements but when the values get collected to a set the equals method of ConfirmCode defines to objects to be equal if they have the same actionEnum. While the first element (which is sorted to the end) has the same actionEnum as the second element in the list, the collector only keeps the first one.