We receive daily patient visit data files from Epic that are imported into Atlas via BrightVine Data Link. Normally the files are received at or shortly after a scheduled time. When they are, we have scheduled processes that do some pre-processing and then import the data into Atlas. Occasionally there are delays on the Epic side that cause late delivery of the files; this usually occurs on the first day of the month because of month end processing, and we are typically notified via email in advance. Because of these occasional delays, we have created a process that reruns the imports if the files aren’t available at the regularly scheduled time.
1 – Data files are delivered
We usually begin receiving the Epic data files at 8:00 AM. They are delivered to the “\\PHSQLAO4600A\EpicData\XXX“ directories as they are produced, with BWH and MGH delivered first followed by the others. Two files are received for each affiliate:
- Patient visits – XXX_PATIENT_IMPORT_WEEKLY_YYYYMMDD.csv. This file includes patient visits with a discharge date of the previous day and looks back seven days. A monthly file of the same name is produced on the fifteenth of the month and includes all visits from the previous 90 days.
- Future visits – XXX_PATIENT_IMPORT_FUTURE_YYYYMMDD.csv. This file includes visits that will occur one year into the future.
2 – Files are formatted, copied, and archived.
The file copy jobs (“Epic File Copy – XXX”) run on PHSQLAO4600A following the schedule listed below (the “Epic file copy job” column). These jobs run a PowerShell script that:
- Corrects formatting issues by removing quotes within fields
- Compares todays Future visit file to yesterday and produces a file that includes only the differences – called “XXX_PATIENT_IMPORT_FUTURE_YYYYMMDD_diff.csv”
- Moves the formatted files to their respective Importfiles directories
- Visits – \\PHSQLAO4600A\ImportFiles\XXX\Epic\EpicVisits
- Futures – \\PHSQLAO4600A\ImportFiles\XXX\Epic\EpicFutures
- Moves the raw data files to the archive folder – \\PHSQLAO4600A\EpicData\XXX\archive
3 – Epic Import Queues run
The Epic imports are run via a scheduled Queue (“Epic Daily Patient Import”) in Atlas following the schedule listed below (the Atlas Queue column). The queue does the following:
Runs the “Clear Epic Future Visits” Global Change. This removes Future visits that are now in the past,- Runs the “Epic Daily Patient Import” BVDL import job,
- Runs the “EPIC Future Visits” BVDL Import job, and
- Archives the data files.
4 – Epic Import Rerun (if needed)
Most of the time we receive the files, they are formatted, copied, and imported in the morning. Occasionally the data files are delivered late and miss the scheduled Atlas Queue time. When that occurs, we have a “File watcher” process (Administration > File watcher) called “Rerun Epic Queue” that looks for the files and then reruns the queue.
| Affiliate / Entity | Files | Epic file copy job | Atlas queue | Queue rerun visit import via file watcher | Queue rerun future visit import via file watcher |
|---|---|---|---|---|---|
| BWH | BWH BWF | hourly 9:00 AM - 4:00 PM | 9:15 AM | every 2 hours 10:15 AM - 6:15 PM | every 2 hours 11:15 AM - 7:15 PM |
| CDH | CDH CDHCVNA | hourly 11:00 AM - 5:00 PM | 11:15 AM | hourly 12:15 AM - 5:15 PM | hourly 12:45 AM - 5:45 PM |
| MCL | MCL | hourly 9:30 AM - 5:30 PM | 10:15 AM | hourly 11:15:00 AM - 5:15 PM | hourly 11:45:00 AM - 5:45 PM |
| MEE | MEE | hourly 10:30 AM - 5:30 PM | 10:45 AM | hourly 11:45 AM - 5:45 PM | hourly 11:55:05 AM - 6:00 PM |
| MGH | MGH | hourly 9:30 AM - 3:30 PM | 9:45 AM | every 2 hours 10:45 PM - 8:45 PM | every 2 hours 11:45 PM - 7:45 PM |
| MVH | MVH | hourly 11:30 AM - 5:30 PM | 11:45 AM | hourly 12:45:42 PM - 5:45:43 PM | hourly 12:55:37 PM - 5:55:38 PM |
| NWH | NWH | hourly 11:15 AM - 5:15 PM | 11:45 AM | hourly 12:30 PM - 5:30 PM | hourly 12:45 PM - 5:45 PM |
| SH | NSMC | hourly 11:30 AM - 5:30 PM | hourly 11:45 AM - 5:45:59 PM | hourly 12:45 PM - 5:45 PM | hourly 12:55 PM - 5:55 PM |
| SRN | SRN SCC SHC SRB HOM | every 2 hours 11:00 AM - 5:00 PM | 11:15 AM | every 2 hours 12:30 PM - 6:30 PM | every 2 hours 1:30 PM - 7:30 PM |
| WDH | WDH | hourly 11:30 AM - 5:30 PM | 11:45 AM | hourly 11:45 AM - 5:45 PM (acquires future visits also) |
Good to Know!
- The EPIC visits – Delete Future Visits Global change is run as part of the nightly Master queue; if it runs earlier in the day, it will delete visits scheduled for that day.
- Even though the Epic files contain much of the same data every day, the import Jobs are designed to never import the same visit twice; any staging data where the PAT_ACCOUNT value is already in the Atlas visits table is deleted. In Epic, PAT_ACCOUNT is referred to as HAR (Hospital Account Record); it’s a value used to group charges for billing purposes. In Atlas, we use it to group multiple encounters into a single visit.