Is master-slave replication my best setup?
I have a big database which writes 1Gb each day. This runs on a low-cost node with just a couple of cores and few Gb memory.
Once or twice per week for a couple of hours, I read from the db and analyse the data.
I wish to pay hourly for the 16vCPU / 64Gb ram node on which to perform these large db reads and computationally intensive analysis.
I seem to require two instances of my database, so as not to have any downtime. (Pausing the write scripts and unmounting the DB volume / mounting it to a high-spec node for 2 hrs is to be avoided.)
My idea is the have master-slave setup, where the master has 100% up-time and it streams to the slave, which is on a volume I unmount weekly for a few hours of analysis.
If the slave is unreachable for a few hours, will this cause problems?
Have I overlooked a superior setup?
See also questions close to this topic
Saiku report error while creating using Mondrian datasource
I am dealing with a Postgresql database to create a new Mondrian DataSource. A relational DataSource works fine adding to the Report Server lib path the Postgresql jdbc client JAR file and configuring database URL, user and password, I am able to create BIRT reports based on this relational DataSources. The problem comes when i try to create a new Mondrian Datasource based on a Star Schema Postgresql Database on the same Postgresql server as Relational one. I have created the OLAP schema with Pentaho Schema Workbench, i am able to navigate with the tool thorugh the Star Schema Database with the jdbc connection. I created the Mondrian Datasource with the following configuration:
Name: Mondrian Username: tad Password: password (the same credentials that i use for relational database conection since both databases are in the same instance) URL: jdbc:postgresql://ip:port/database_name(I cant share ip and database name as per policy ) Properties: type=OLAP name=Mondrian driver=mondrian.olap4j.MondrianOlap4jDriver jdbcDrivers=org.postgresql.Driver Schema: (Schema generated with Pentaho Schema Workbench)
When I click on apply i am not getting any Error (there is no test connection so i guess there is no reason why i should get any error) , but once i tried to create a new Saiku report using this Datasource, i get the following login failed error: The server requested password-based authentication, but no password was provided.
How to avoid duplicate records with user edits in a referential system?
I am curious to see how the SO Community deals with this common database problem.
The problem is this. I have textual descriptions of various problems being entered at the desktop into the table ICD_Descriptions (the middle box below). Generally speaking there will also be one code associated with each description. However, over time (i.e., years), the codes for a specific text phrase/description will change. Hence, a general many-to-many relationship will exist for some codes to descriptions. So a third table, dx_log, is being used to allow for a many-to-many relationship between the code and the descriptions. Lastly, other "children" tables that need to see a specific combination of 'code-description' will be given a reference to the primary key (the recid) of the dx_log. I believe this arrangement to be fairly standard database management.
O.K., now for the problem. I wish the codes in the icd_code table to be unique to that table. I also wish the descriptions in the icd_description table to be unique to their table.
The problem. This being a referential system, changes to either the "data" part of the code table, (the code), or the description (in the descriptions table) will be seen in the child tables.
But, how to correctly manage user edits to a code or description that would conflict with the "unique" rule of the respective table?
As an example, lets say the initial text of a description is "this is mispelled", whereas another description text has "this is misspelled". At one point it time, both phrases coexist and are unique. However, at a later point in time, the wrong record is corrected (i.e., mispelled --->misspelled). When the edits are attempted to be saved to the file, it will be detected that the correct record already exists.
So if the dx_log table is also unique on (icd_description_recid, icd_code_recid), then simply replacing the reference to the already existing correctly spelled record in the dx_log will lead to a violation of uniqueness for the dx_log table. Therefore, I can think of only three solutions:
- The table which the children tables actually reference CAN NOT HAVE A UNIQUES constraint on its reference pointers,
- Leave the uniqueness constraint on the dx_log, but when a conflict would violate uniqueness, then use a migrate procedure to move the child table references to the existing record (in postgresql this would be heavy use of the catalogs) to the "new record", then delete the existing record before adding the new record.
- Add an additional self-reference pointer in the dx_log such that when a record would conflict with an already existing record in dx_log, then don't change it but rather place a pointer to the already existing "correct" record in the dx_log.
I hope I've explained my question well enough. What is the recommended approach?
Thanks for any comments.
ASP.NET MVC4, EF 6, Postgresql : script generation issue
I have been developing ASP.NET MVC 4 application using PostGreSQL database.
- "Npgsql" - Npgsql is the opoen source .NET data provider for PostgreSQL.
- "Npgsql for Entity Framework 6" - PostgreSQL provider for Entity Framework 6.
Below exception occurs when calling the SimpleMembership provider.
2018-01-19 09:00:30 IST : [1-1] user=postgres,db=PatientScheduler,app=[unknown],client=127.0.0.1 ERROR: syntax error at or near "[" at character 14 2018-01-19 09:00:30 IST : [2-1] user=postgres,db=PatientScheduler,app= [unknown],client=127.0.0.1 STATEMENT: CREATE TABLE [UserProfile]([UserId] int NOT NULL PRIMARY KEY IDENTITY, [UserName] nvarchar(56) NOT NULL UNIQUE)
[PostgresException (0x80004005): 42601: syntax error at or near "["] Npgsql.NpgsqlConnector.DoReadMessage(DataRowLoadingMode dataRowLoadingMode, Boolean isPrependedMessage) +311 Npgsql.NpgsqlConnector.ReadMessageWithPrepended(DataRowLoadingMode dataRowLoadingMode) +272 Npgsql.NpgsqlConnector.ReadExpecting() +29 Npgsql.NpgsqlDataReader.NextResultInternal() +296 Npgsql.NpgsqlDataReader.NextResult() +184 Npgsql.NpgsqlCommand.Execute(CommandBehavior behavior) +906 Npgsql.NpgsqlCommand.ExecuteNonQueryInternal() +134 WebMatrix.Data.Database.Execute(String commandText, Object args) +129 WebMatrix.WebData.SimpleMembershipProvider.CreateTablesIfNeeded() +309 WebMatrix.WebData.WebSecurity.InitializeMembershipProvider(SimpleMembershipProvider simpleMembership, DatabaseConnectionInfo connect, String userTableName, String userIdColumn, String userNameColumn, Boolean createTables) +86 WebMatrix.WebData.WebSecurity.InitializeProviders(DatabaseConnectionInfo connect, String userTableName, String userIdColumn, String userNameColumn, Boolean autoCreateTables) +88 PatientSchedulingSystem.Filters.SimpleMembershipInitializer..ctor() in d:\Projects\PatientSchedulingSystem\PatientSchedulingSystem\Filters\InitializeSimpleMembershipAttribute.cs:41
The Entity Framework is not generated scripts based on PostgreSQL standard.
Please assist on this issue. Thanks in advance.
Do all the masters work at the same time or only one work for cluster about K8S HA?
If install k8s cluster with 2 masters for HA, are they all work at the same time or only one work?
If use the official way:
org.apache.jorphan.util.JMeterError: Could not return sample
I am running Jmeter test in distributed mode. I have set up SSH tunneling for slaves since they are not in same region. I have executed 2700 users which ran fine. When we try to run 5200 users, users are going into finished status although steady state is 1 hours. I am using ultimate thread group. 4500 users are running fine and 600 Users are going into finished state out of 5200 usesr.
Seeing below error in Jmeter Server logs:
ERROR o.a.j.t.JMeterThread: Test failed! org.apache.jorphan.util.JMeterError: Could not return sample at org.apache.jmeter.samplers.StandardSampleSender.sampleOccurred(StandardSampleSender.java:70) ~[ApacheJMeter_core.jar:3.3 r1808647] Caused by: java.rmi.ConnectException: Connection refused to host: 127.0.0.1; nested exception is: java.net.ConnectException: Connection refused: connect at sun.rmi.transport.tcp.TCPEndpoint.newSocket(Unknown Source) ~[?:1.8.0_151]
Any idea? What is causing this?