Freelancer: RedLayers
Kongsi:
Laporkan Entri

Best way to do it

Hello, So I've proudly completed this in the most efficient way there is. Here's a Demo Video: https://youtu.be/vKaNR1o5Zq4 The way this works is by executing "Download All" first and then parsing that PDF file and retrieving whether the VALID entries are Qualified or Not. It's very fast and very efficient since it uses something that we already have instead of over-complicating things, so it's a lot more optimized in general. Also, using this, we gained a new feature: If the user clicks "Download All" or "Extract All to CSV" then clicks ANY of them again he will NOT have to wait at all; it'll instantly happen. Thanks for reading, Mario Adabachy.

Penyertaan Peraduan #2 untuk                                                 Data Export to CSV from Web Site

Papan Penjelasan Umum

  • a365guy
    Penganjur Peraduan
    • 5 tahun yang lalu

    Hi Mario - I'm uploading an attachment with US based Starbucks addresses for you to use on the test. I'm also uploading a picture of the Congressional details I was talking about in bullet 7 earlier. This data is on the web site, not in the PDF.

    1. Test your build against the addresses provided

    • 5 tahun yang lalu
  • a365guy
    Penganjur Peraduan
    • 5 tahun yang lalu

    2. There seems to be a massive performance increase when I run batches of 200 addresses compared to more and then failure at 400 or so addresses. How can we allow for an upload of thousands of addresses, but the tool just runs them in batches of 200? I know we're already sort of doing that but in batches of 6 which is how the progress bar works. Just FYI, I'm looking to use this tool to run against tens of thousands of addresses eventually covering 300,000 addresses.
    3. Take a look at the attached screenshot where I see the extra details on the web site's data. How can I grab that and bring it back into the final CSV?

    I think once these are done, that's a wrap and I'll award it after doing a full test in my environments.

    Thanks, Neil

    • 5 tahun yang lalu
  • RedLayers
    RedLayers
    • 5 tahun yang lalu

    New Update: https://youtu.be/3hUMcJ-JA-o
    Let me know if there's more to be done.

    • 5 tahun yang lalu
    1. RedLayers
      RedLayers
      • 5 tahun yang lalu

      Since you didn't reply I thought today I'd just 2 more columns, so now we have 3 extra columns than the original data which are Qualified/NotQualified,Census Tract,County. Obviously if we're not qualified Census Tract and County are empty.

      Before:
      Name,Address,City,State,Zip
      Starbucks,,Crescent City,CA,
      Starbucks,,Riverview,FL,
      Starbucks,,Sebring,FL,

      After:
      Starbucks,,"Crescent City",CA,,"Qualified HUBZone","Tract ID: 06015000101 - County: Del Norte County - State: CA","County ID: 06015 - County: Del Norte County - State: CA"
      Starbucks,,Riverview,FL,,"Not Qualified",,
      Starbucks,,Sebring,FL,,"Qualified HUBZone","Tract ID: 12055961100 - County: Highlands County - State: FL"

      • 5 tahun yang lalu
    2. RedLayers
      RedLayers
      • 5 tahun yang lalu

      Latest Update - 2minutes video: https://youtu.be/v-YIOjcRE9I

      • 5 tahun yang lalu
  • RedLayers
    RedLayers
    • 5 tahun yang lalu

    Alright, I got it, should be done in few hours.

    • 5 tahun yang lalu
  • a365guy
    Penganjur Peraduan
    • 5 tahun yang lalu

    6. Add column to export CSV for "Status" with options: Qualified, Not Qualified, Unknown
    7. Add column to export CSV for "Congressional District" pulling data from the PDF (right below where it says qualified)
    8. On both downloads (PDF or CSV), when the button is clicked a typical Windows dialogue box pops up to allow a user to save?
    9. We'll want to test this new version against a lot of addresses. Please use US based Starbucks store locations to test - I recall grabbing those addresses for testing on the first version.
    Let me know if you have any questions.

    • 5 tahun yang lalu
  • a365guy
    Penganjur Peraduan
    • 5 tahun yang lalu

    Watching the video again and some feedback based on our comments:
    1. Let's remove the "Export as CSV" column from the GUI (we won't ever export just one or a few)
    2. "Export All to CSV" button is as expected
    3. Change the "Download All" button to "Download Combined PDF"
    4. Keep "failed" rows in export CSV (helps us know there's problems and not get confused by data mismatch)
    5. Export CSV should basically be identical to the uploaded CSV - with new columns added

    • 5 tahun yang lalu
  • RedLayers
    RedLayers
    • 5 tahun yang lalu

    Thanks again for the appreciation, I simply consider myself as part of the team and make sure I plan things and get them done the best way possible, and I'm very glad that "my" best way was also "your" best way.

    I think I'll send you the source code tomorrow morning even before you award me, I don't really mind, maybe you can get your developer to take a look at it and confirm its quality if needs be.

    Cheers,
    Mario Adabachy.

    • 5 tahun yang lalu
  • RedLayers
    RedLayers
    • 5 tahun yang lalu

    Firstly, thanks for appreciating good work, Neil.
    Secondly, I am modifying the version you uploaded in the contest, I simply edited the files accordingly and made sure we added the "Export to CSV" feature without adding extra messy functions or degrading the performance of the whole project.
    About the CSV data, that is exactly what we're doing. If you check times 0:17 and 2:18 if the video, as you can see we simply add an extra column that says Qualified or not Qualified. So again, we're doing exactly what you have in mind.
    The only difference here is that the rows that failed are ignored instead of adding "unknown" but we can SO easily make that happen. Don't worry about it.
    We're only using 1 extra button that says "Export all to CSV" which is necessary; the user choses whether he wants to Download All as a PDF(old feature) or Export all to CSV(new feature).

    • 5 tahun yang lalu
  • a365guy
    Penganjur Peraduan
    • 5 tahun yang lalu

    aarrrgg
    Second, Let's discuss the code / version 2.0. Are you modifying version one with the new functionality? That's what I need and what it seems like you are doing. Just want to clarify.
    Third, let's talk about the data you are exporting to the CSV. I would like all addresses to push to a CSV with the same rows and just new columns. One of the columns would say qualified, not qualified, or unknown. The "unknown" would tell me that row failed.
    Fourth, why are you using new buttons? I couldn't figure that out?
    Keep up this good work, and I'll award it to you. While others are asking me questions, you just went and did it. That's what I look for from someone who is experienced and "gets" it. Thanks, Neil

    • 5 tahun yang lalu
  • a365guy
    Penganjur Peraduan
    • 5 tahun yang lalu

    Hi Mario - great job being fast. I have another project behind this one, and it will go your way if this can be done to my satisfaction.
    First - great video. Very helpful for me to see that you "get" what I'm talking about.

    • 5 tahun yang lalu
  • RedLayers
    RedLayers
    • 5 tahun yang lalu

    Can I please get some feedback? I'm pretty sure I nailed everything perfectly

    • 5 tahun yang lalu
  • RedLayers
    RedLayers
    • 5 tahun yang lalu

    I saw your comment and I believe I am already doing all that

    • 5 tahun yang lalu

Tunjukkan lebih banyak komen Sedang diproses...