Data Processing and analytics - Production environment

Hi, Once you share db & required processing / analytics you need we will be able to confirm on time & estimate. We have got 14 years of experience in building diff. type and size of db driven apps & reporting/analytics solutions. Here are answers, 1. We will use SQL with Python for better logical implementation for processing. 2. We can use joins but again once you share specific details we will be able to comments better. As sometimes approach needs to be defined based on performance diff. 3. Needs to maintain as much low load on db server as possible. Perform operation on backend if possible as compare to db engine 4. I guess this is very specific and needs to be addressed with case. Usually as i have suggested it will go based on point 3 Thanks, CWE

Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. Please visit : https://www.freelancer.com/u/schoudhary1553 I have excellent command over English. I am a hard worker, productive and worthy of your attention I hope, I would be the right candidate for this post. Awaiting an affirmative response from you. Kinds Regards, Sandeep

Hello, Dear How are you? I have read your project description and am ready for discussing with you. I have experienced in developing Amazon Web Services, Data Processing, Database Administration, MySQL, Python. I will work very hard and best for you. Best Regars

Hello sir I am a senior python developer with 9 years of experience. I am familiar with Sqlalchemy and AWS RDS. I am suitable for this project and ready to start the work. Best Regards, Yongtao

1. I think python would be the best for it. 2. a little not sure what means. But I think we can query once time, after that, we can loop. 3. What are the top 3 things to consider while processing data in production environment? 4. We will split them such a few rows etc. Hi There, I have gone through the job details carefully and I am excited to start working on this project. I am ready to start work immediately. The budget can be negotiated, so that we can discuss again if you want. Please let me know if you are interesting me, so that we can have a good idea about the projects. Looking forward to hearing from you son build up a long term professional relationship with you.

Hello, sir I've read all of your requirements extremely carefully. This is the one of a team experienced in PHP, MySQL and Javascript. It will be pleasure to work with this project. I'd like to discuss in detail over chat for making sure that we can finish this job successfully. Thanks, Regards.

Hi! How are you? I am python Expert. I can help you. I can deliver your tasks with great quality If you believe me and I will do my best for you. Please check my profile and message me for more details. Thanks

Hello Ronak K., After reviewing your project I believe that I would be a great fit as the skills mentioned: python, data processing, mysql, amazon web services, database administration are all areas I am familiar, proficient, and confident in. I have worked with companies ranging in size from start-ups, to 7-figure revenue corporations and am comfortable on projects of any size, or duration. I am able to provide a list of references, and a portfolio of past work per your necessity- furthermore being a NYC local I am able to facilitate any form of skype/video/voice call at any point. I would love to chat more about your project, and hope to have a working future together soon. Thanks.

Hello, Starting from answers: 1. I vote for scala, but it's just because of personal preferences. Other than that it doesnt matter at all 2. Loop over result set, use bidirectional cursors if driver supports, use higher order collection functions. Any of those methods 3. #1 Use test DB instead of production #2 Know your data #3 Test ur code 4. Most of the db drivers support various query timeouts settings, it will help to avoid endless queries. For generic processing its a matter of type of processing required. Most generic strategy is to use async parallel processing with callbacks. Similar to whats done in akka framework. But for simple workloads old good catch and retry will be enough =) Let me know if you're interested in cooperation. Regards, Mykhail

I am a professional data scientist based in Scotland, I have worked on various data analysis projects I will be more than happy to further discuss your project.

Hello there, Hope you are doing good...!!! As per your requirement, I am very much suitable for the job kindly invite me for a chat for further discussion regarding the project. I have more than 6 years of experience in web & mobile app development you will get good quality work from me. Thank you

Hello, I would love to do your task, I could handle this task as soon as possible within your time given. Please see my detailed proposal and samples attached. I'm glad to answer any questions you may have . Thank you for your time, Nur hanis

I will do it better than you want it , I will do it by perfection...

Hi,I have read your requirements of Data Processing and analytics - Production environment and I’m pretty confident that I will be able to get your work done within the given time your requirements conclude. I am very keen to discuss further. You can reach me at any time on online. I have also a few clarifying questions on the project. I’m an Expert, Honest, Sincere, Quick learner and Responsible worker as I’m taking my work like my Responsibility. kind regards.

I have over 14 years in Electronic data processing and analysis and programming SQL with ASP.net and PHP

I'm BI & Analytics Expert, i have been working in Enterprises in Middle east and Spain. I believe i can do your project, here are my answers to your questions 1. Python would be a good option, in order not to make unnecessary load over the server. 2. we have here 2 choices: -if we will work with PYTHON, then no need to run a query in loop, we will have datasets derived from the tables (based on the dates we need) and working on them to produce the needed variables -if we will have to work with SQL, then we can use intermediate tables for our calculations to avoid any kind of loops, also in this way we won't redo any calculations as it will be save in intermediate tables. 3. - First we should pick the time where the server is not accessed often to do the processing, for example from Midnight to 6 AM - There should be a checking points while processing, so in case of any failure during the process, we resume from a checked point and not redo the whole processing. - Never change the data, just add new data (unless recovery is required) 4. We should start the query/process on the secondary instance. to avoid more delay. then we check the SQL of the hung query before killing its process, to understand why it did hung, was the reason from the query ? or the reason is from the server ? or the reason could be the table(s) used in the query were having a lock due to inserting of data?

Good Day! Dear Ma’am/Sir: For starters, I have all the skills and experience listed in the job description, and I'm confident that I can make an immediate impact on your company. If chosen for this role, I will continue to deliver high-quality work for the continued success of your organization. Sincerely yours, Joshua Lebajo Applicant

I am very good in data intery .I completed my work before deadline..my main moto is to make customers happy with my hqrdwork and with my skills Relevant Skills and Experience Data entry Autocad Microsoft word Powerpoint

I am working almost 20 years with applications that are interacting with Relational Databases within large organizations with millions of subscribers/clients. 1. The implementation language is a combination of what is in place, the knowledge that exists in the team and other things that are happening in the project. I would suggest that a combination of both is ideal, since some things can be done more efficient in Python than MySQL (SQL). e.g. NLP processing, set up REST API, object oriented design (Python comes with SQL Alchemy...) 2. a. SQL joins b. Use Pandas to extract the tables with the build in driver for SQL and then do some manipulation including joins of the datasets. 3. a. Fault tolerance. e.g. smart use of transactions in order to keep the data consistent in the case that something goes right. b. Ability to keep track of what is happening. It is auditing, but in some cases it is covered by detailed logging information.. (not debug, but enough details for the production engineers to understanding immediately where the pain is).. c. Safety... 4. a. It depends on the application. if the response is required immediately, then the SQL query should run in a separate thread with a timeout defined. In these system, bringing the data in time is also important. b. Logging... alerts from a supervisor process

I'm good in data administration