Receive messages/events from SQS and map them to Dynamodb

See also questions close to this topic

• Read each name in Array list to create seperate object for

  I have a file that has student names, age, and an id number. I have a student class that holds the everything above for each student object. I stored all the names, id numbers. and age separately in an array list. Now im trying to assign the info to create a student object.

  public class Student {
      private String lName;
      private int idNumber;
      private int age;
  
      public Student() {
          lName = "";
          idNumber = 0;
          age = 0;
      }
      public Student(String l, int i, int a) {
          lName = l;
          idNumber = i;
          age = a;
      }
  
      public void setName(String last) {
          lName = last;
      }
  
      public String getName() {
          return lName;
      }
  
      public void setIdNum(int num) {
          idNumber = num;
      }
  
      public int getIdNum() {
          return idNumber;
      }
  
      public void setAge(int a) {
          age = a;
      }
  
      public int getAge() {
          return age;
      }
  }
  

  My Text File looks something like this: This info is stored in parallel array lists. I don't quite get how to implement this into an object to pass it into my second contractor method.

    Josh          2134         19
    Smith         5256         21
    Rogers        9248         19
    Andrew        7742         20
  

  Here's what I've tried;

  public static void main(String[] args) {
      String file = "studentData.txt";
      Scanner reader = new Scanner(file);
  
      ArrayList<String> lastNames = lNames(file);
      ArrayList<Integer> idNumbers = idNum(file);
      ArrayList<Integer> ageList = ages(file);
  
      Scanner input = new Scanner(System.in);
  
      Student s1 = new Student();
  
      // confused about how to implement this constructor with the textile info
      for (int i = 0; i<idNumbers.size(); i++) {
          Student user = new Student(lastNames.get(i), idNumbers.get(i), ageList.get(i));
      }
  
      //user enters idNumber to display age
      System.out.println("Enter ID Number"); //exception handling to be added
      int idNum = input.nextInt();
  
      for (int i = 0; i<idNumbers.size(); i++) {
          if (idNum == idNumbers.get(i)) {
              s1.setAge(ageList.get(i));
              System.out.println(s1.getAge());
  
          }
      }
  }
  
• Using EdittextPreference for Goto search

  sorry for my poor English. I want to use EditTextPreference in my bottom nav like the pic below, ![screenshot][1]

  I have recycleview xml in my project with in many sub cardview layouts(which is scrollable) and I want to create item in the bottom nav which is called "Goto". When the "Goto" item is clicked i want it to pop-up like the screenshot. And when user enters a number(according to the cardviews i.e if the number of cardview is 40 user must enter 1-40) I want to search the cardview by its ID. Thank you and I hope u got it, If u have any questions let me know [1]: https://i.stack.imgur.com/grK8P.jpg

  My xml format look like this. As you see in the blow since the cardviews are huge in number it is not cool to scroll all the way down that is why i need Goto item in the bottom nav to search it by its ID when the user click number in the EditTextPreference as u see in the screenshot. i.e The screenshot is not from my app

  <LinearLayout>
   <LinearLayout>
    <androidx.cardview.widget.CardView>
      <RealtiveLayout>
         <Textview/>
      <RealtiveLayout>
   </androidx.cardview.widget.CardView>
   </LinearLayout>
  
   <LinearLayout>
    <androidx.cardview.widget.CardView>
      <RealtiveLayout>
         <Textview/>
      <RealtiveLayout>
   </androidx.cardview.widget.CardView>
   </LinearLayout>
  
  <LinearLayout>
    <androidx.cardview.widget.CardView>
      <RealtiveLayout>
         <Textview/>
      <RealtiveLayout>
   </androidx.cardview.widget.CardView>
   </LinearLayout>
  
    <LinearLayout>
    <androidx.cardview.widget.CardView>
      <RealtiveLayout>
         <Textview/>
      <RealtiveLayout>
   </androidx.cardview.widget.CardView>
   </LinearLayout>
   
    ..
    ..
  
    ..
    ..
    many more..
  
• How to get remaining time of the day in java?

  I would like to calculate the time remaining for next day 00:00:00 from the current date time.

  For e.g. time difference between 2022-05-07T05:49:41.883807900Z and 2022-05-08T00:00:00Z

  Expected answer: 18:10:19 or 65419 (in seconds).

  How can I achieve this with efficiently using java 8?
• Upload file from html when block public access is true

  I am using django-s3direct to file upload

  https://github.com/bradleyg/django-s3direct

  Using IAM role setting because I upload the file from the server on ECS container.

  Now I set the blockPublicAccess of S3 false.

  When uploading images from html, there comes error.

  https://s3.ap-northeast-1.amazonaws.com/static-resource-v/images/c64d6e593de44aa5b10dcf1766582547/_origin.jpg?uploads (403 (Forbidden)
  )
  
  initiate error: static-resource-v/line-assets/images/c64d6e593de44aa5b10dcf1766582547/_origin.jpg AWS Code: AccessDenied, Message:Access Deniedstatus:403
  

  OK, it is understandable.

  Browser try to access the for initiation.

  However there is any way to upload file from browser when blockPublicAccess is true??
• Linux on Lightsail instance is asking for a password and it's not working

  I'm trying to restart mariaDB on Ubuntu but it's not letting me.

  I enter:

  systemctl restart mariadb
  

  and get:

  ==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
  
  Authentication is required to restart 'mariadb.service'.
  
  Authenticating as: Ubuntu (ubuntu)
  
  Password: 
  
  polkit-agent-helper-1: pam_authenticate failed: Authentication failure
  
  ==== AUTHENTICATION FAILED ===
  

  I have the same password for all functions so I do not understand why it is not working. What can I do?
• AWS Pinpoint sendMessages() Addresses param field error

  I'm having trouble replicating the format of the params object's Addresses format in a way where I can easily add to the object.

  If I use this as the params with destinationNumber[0] and destinationNumber[1] in the format of 1 + 9 digit number ie 13334535667 then it sends the message to both numbers no problem.

   const params = {
          ApplicationId: applicationId,
          MessageRequest: {
              Addresses: {
                  [destinationNumber[0]]: {
                      ChannelType: 'SMS'
                  },
                  [destinationNumber[1]]: {
                      ChannelType: 'SMS'
                  }
              },
              MessageConfiguration: {
                  SMSMessage: {
                      Body: message,
                      Keyword: registeredKeyword,
                      MessageType: messageType,
                      OriginationNumber: originationNumber
                  }
              }
          }
      };
  

  I'm trying to replicate this format for Addresses, but I'm getting Unexpected key '13334535667' found in params.MessageRequest.Addresses['0']. The format my console output shows for Addresses is

  [
    { '12345678910': { ChannelType: 'SMS' } },
    { '12345678911': { ChannelType: 'SMS' } }
  ]
  

  I'm using a map to call this

  function createPhoneMessagingObject(phoneNumber: string) {
      return {
          [phoneNumber]: {
              ChannelType: 'SMS'
          }
      };
  }
  

  I tried wrapping key in array like in phone object, but per the output, the brackets goes away so maybe there's an easier/more correct way of doing this. I appreciate any help!
• Amazon DynamoDB delete data based on partition key

  I am trying to delete multiple records in DynamoDB table using the partition key and sort key. The approach is taking lot of time to delete the records. I am following below approach implemented using java aws sdk library.

   public static ItemCollection<QueryOutcome> getKeyDataFromTable(String tableName, HashMap<String, String> keyValue){
          QuerySpec spec = null;
          //Connect to Database Table
          spec = new QuerySpec().withKeyConditionExpression("pKey" =:v_id").withValueMap(new ValueMap().withString(":v_id",keyValue.get("partitionkey")));
          ItemCollection<QueryOutcome> items = table.query(spec);
          } 
  
  
  
   public void deleteItems(){
      String tableName="tableName";
      String partitionKey="partitionKey";
      HashMap<String, String> keyValue = new HashMap<>();
      keyValue.put("partitionKey",partitionKey);
      ItemCollection<QueryOutCome> keyDataFromTable = getKeyDataFromTable(tableName,keyValue);
      IteratorSupport<QueryOutcome> iterator = keyDataFromTable.iterator();
      Item item=null;
      List<String> sortKeyList = new ArrayList<>();
      while(iterator.hasNext()){
      item=iterator.next();
      sortKeyList.add((String) item.get("sortKey"));
      }
      for(String skey:ortKeyList){
      DeleteItemSpec deleteItemSpec = new DeleteItemSpec().withPrimaryKey("partitionKey","partitionKey","sortKey","sortKey");
      table.deleteItem(deleteItemSpec);
      
      }
      }
  

  Is there any right approach of doing this dynamically with better performance?
• How to scan a table in DynamoDB with time period?

  I have a table in DynamoDB and it has an attribute 'createDate' and I want to do a scan using a filter in a specific period of that attribute (for example: 2022-01-01 to 2022-01-31) but I don't know exactly if it's possible and how to do. If anyone has done this and can help me it would be very helpful.

  just one more question: is it possible to put the result in a CSV file?

  Here is my code where I can scan with a single date:

  public class QueryTableResearchAnswers {
  
      static AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard().build();
      static DynamoDB dynamoDB = new DynamoDB(client);
  
      static String tableName = "research-answers";
  
      public static void main(String[] args) throws Exception {
  
          String researchAnswers = "Amazon DynamoDB";
  
          findAnswersWithinTimePeriod(researchAnswers);
          //findRepliesPostedWithinTimePeriod(researchAnswers);
      }
  
      private static void findAnswersWithinTimePeriod(String researchAnswers) {
          Table table = dynamoDB.getTable(tableName);
  
          Map<String, Object> expressionAttributeValues = new HashMap<String, Object>();
          expressionAttributeValues.put(":startDate", "2022-01-01T00:00:00.0Z" );
  
          
          ItemCollection<ScanOutcome> items = table.scan("createDate between > startDate", // FilterExpression
                  "bizId, accountingsessionid, accounttype, acctsessionid, choicecode, contextname, createDate, document, framedipaddress," +
                    "macaddress, macaddressnetworkdata, machash, mail, nasgrelocalip, nasidentifier, nasipaddress, nasportid, network, networktype, networkuuid, phone," +
                  "question, questionanswer, questioncode, realm, relayingmacaddress, remoteipaddress, useragent, username", // ProjectionExpression
                  null, // ExpressionAttributeNames - not used in this example
                  expressionAttributeValues);
  
          System.out.println("Scan of " + tableName + " for january answers");
          Iterator<Item> iterator = items.iterator();
          while (iterator.hasNext()) {
              System.out.println(iterator.next().toJSONPretty());
          }
      }
  
• What Database should I use?

  I'm working on a project and I'm quite confused what the best DB is to use as the requirements are quite niche. Some of the requirements are: How can I meet these requirements?

  1. To be able to run queries such as COUNT of unique values (leaning toward SQL)
  2. Flexible Schema as the records will often be similar in some KVP/Columns but different in others (leaning towards noSQL)

  I was thinking of using MySQL or PostgreSQL but the schema will change as the users use the application.

  NoSQL is something I have also explored with the likes of DynamoDB and it fits perfectly apart from the cost to run Scans to find distinct attributes and sometimes return lots of data.

  Finally, I was thinking of a DynamodDB -> Glue -> RDS (and use RDS to query the data) which I thought could get quite messy and even DynamoDB -> ElasticSearch but not very familiar with the concepts and how I could make good use of them.

  I'd be grateful if anyone has any thoughts or similar challenges and can point me towards the right direction.

  Have a nice weekend! :)
• Is there a way to filter with AWS SNS filter policies from the message body to SQS queues?

  I have a general question about SNS filter policies. I know that the filter policies filter based on the Message Attributes. If we wanted to filter based on the body, is there a work around to do so?

  The SNS topic will be delivering different types of data to SQS queues based on the filter policies.
• S3 : Put Vs Multipart Notification clarification

  Wanted to confirm the behavior I am seeing. I am using Amplify.Storage.uploadFile to upload a file. The file being uploaded can be of any size . Seems like the amplify sdk decides the mechanism for upload depending upon file size. I am listening to SQS notifications on upload. This is the behavior that I see

  Enable only Multipart upload complete notification:

  1. Smaller file size : receive nothing
  2. Larger file size: receive single ObjectCreated:CompleteMultipartUpload event

  Problem: I miss out on the smaller file.

  Enable both PUT and Multipart upload complete notification

  1. Smaller file size: receive put event
  2. Larger file size : get multiple ObjectCreated:CompleteMultipartUpload events

  Problem: I don’t know which of the notifications to listen to for the larger file size. I don’t know if anything is guaranteed about the timing of the multiple notification. Can I assume to try to read the file and if the multipart upload has not truly finished then I wouldn’t be able to download the file and hence I can ignore the notification ?

  Thoughts ?
• Lambda trigger again after timeout even sqs visible timeout is same as lambda timeout

  Lambda has triggered again even after the SQS visibility timeout is the same as lambda timeout around 8min.

  Having SQS visibility timeout greater than lambda timeout, inflight messages in sqs were being picked up again in lambdas.

  is there a workaround?

  we dont have DLQ.