Contact
Channels:  Pm  P-C  CH  HCw  Hmii  Mag  Lasco  |  Blog 
 
P.A.Semi

Heliograf API


Images in Heliograf are stored in daily Zip files, updated once daily... (which may differ in NRT channels, to be implemented later...) Images are available in various sizes, usually 1024px, 512px, 160px, 80px, and daily grids with tiles 160px or 80px, and monthly grids one image daily...


While SDO / AIA makes one image in 4K resolution for each channel each 12 seconds, making literally petabytes of data in FITS format, and NASA / GSFC provides "Browse" jpegs one for every ten minutes, and JSOC provides 1K Fits files every 2 minutes, here in Heliograf there is quality-checked selection for fast browsing, resized to 1K resolution or 512px usually from 2K Jpegs or 1K Fits files, some channels have 8 images daily in 3 hour step, some 24 images daily in 1 hour step, in some channels when there is something interesting happening, it may be sub-divided into 20 minutes or 10 minutes step...

The Channels here are usually post-processed and combined from multiple source channels, as described in individual Channel pages... (maybe sometimes later there will also be unprocessed original jpeg channels for 171,193,304 channels...?)

List of available Channels is below ...

Listing of available images is available usually in Simplified JSON format - names are not quoted unless necessary... Since many browsers insist on quoting names, you can use javascript routine jsonParse() for input and jsonStr() for output, available in dynui52.js, or javascript eval(jsonText) ... (Similar routine for PHP is available on request...) 


Every "Channel" has its front-end in a virtual "sub-dir" in /Hg , with lowercase channel code following, like /Hg/ch/ is html front-end component, individual images can be accessed via various url links below that path, like /Hg/ch/20230731_1500_Mix171_193_1024px.jpg:nearest , and listings from urls like /Hg/ch/listing/20231101.json ... (as described in more detail below in Comments section...)  


The Channel also supports browsing by (virtual) directories:

You may try: /Hg/ch/2023/ /Hg/ch/2023/07/ /Hg/ch/2023/07/31/ leads to /Hg/ch/2023/07/31/20230731_1602_Mix171_193_1024px.jpg which is available regardless of actual sub-dir below the Channel path ...
Actually it is stored on server in dataPath/CH/2023/20230731_CH.zip , also available for download, as described below...


The user interface html accepts time code in url "search" specifier /Hg/ch/?230731_1600 to open on specified time...


Described in more detail in Comments section below...


(older first) older first
newest first
neither

Comments:

P.A.Semi #1
Remarks, items:

chanRanges.json described below in #7 shows available ranges of data in all channels...

In the following examples, "ch" is code for channel CH (CoroHole)...

/Hg/ch/20230731_1500_Mix171_193_1024px.jpg:nearest
20230731_1500_...:nearest - suffixes can be :nearest :next :prior (or :prev) ...
/Hg/ch/?latest  is html browser with latest image
/Hg/ch/latest.jpg  with x-filename: 20231119_2302_Mix171_193_1024px.jpg
/Hg/ch/latest_512px.jpg -  same, smaller image...
/Hg/ch/latest.json - monthly json, now (22nd) cca 150kb for channel CH...
/Hg/ch/listing/20231101.json - daily json listing...
/Hg/ch/listing/20231101.lst - simple file-name list...

X-FileName: header - mangled lowercase by nginx ...
x-filename: 20231119_2302_Mix171_193_512px.jpg

X-PosInfo: {...} - mangled lowercase by nginx ...
x-posinfo: {lon:61.88,lat:2.332,car:2277,dist:212.38198375799,rm:481.70822,ra:572.98422}
{lon,lat,car,dist,rm,ra}
lon = Longitude, lat = Latitude, car = Carrington rotation, dist = Distance in Solar radii, rm: solar radius pixels on Magnetograms, ra: solar radius pixels on AIA images (1K resolution)
rm * 1.199/1.008 = ra
Radius on Magnetograms (rm), Carrington position (lon,lat,car) and Distance are fetched from NRT magnetogram Fits files from JSOC...

These values (with Center X,Y ?!) may be used to project the image onto (for ex. rectangular) map of Sun...
Adding {cx,cy} with disk Center position would require modifying all zips on the server...
(In some image formats it is available, either calculated during processing and verified by rm, or from fits sources in channels produced from fits files... But it is not present in json files packed in daily zips... So it would be added sometimes later...) 
(Images in most channels are already centered, with 1024x1024 images the center of Solar disk should be 512,512 ... Some channels may be centered only "usually"... During eclipse seasons usually in February and August the SDO jumps up/down by a pixel or few between 6 and 9 am, and occasionally it "jumps" in other times, while on magnetograms and intensitygrams it is often already compensated, or the affected images are missing altogether...) 

...
P.A.Semi #2
latest.json - monthly listing:
All times are UTC (sometimes maybe TAI? The difference cca 37 seconds (not sure now precisely) is usually negligible...) ...

Using Simplified JSON format - names are not quoted unless necessary...

{dtFrom:"2023-11-01",dtTo:"2023-11-19",
 Items:[{
  Time:"2023-11-01T00:02:00",
  File:"20231101_0002_Mix171_193_1024px.jpg",
  SrcFile:"20231101_0002_Mix171+193.jpg",
  Width:1024,Height:1024,
  Lon:311.27302,Lat:4.43711,Car:2277,Dist:213.35994725089,
  MagRadius:479.50368287173,
  gi:"#0",ix:0,iy:0,iw:160,ih:160,
  DayIco:"DayIco160/20231101_CH_DayIco160.jpg",
  FileIco:"Ico160/20231101_0002_Mix171_193_160px.jpg"},
  ...
 {Time:"2023-11-19T22:58:32.100",
  File:"20231119_2302_Mix171_193_1024px.jpg",
  SrcFile:"20231119_2302_Mix171+193.jpg",
  Width:1024,Height:1024,
  Lon:61.35989,Lat:23.25104,Car:2277,Dist:212.38806594371,
  PAngle:-0.0134429932,Cx:511.603088,Cy:511.513245,
  RadAng:971.17303466797,
  Source:"hmi.M_720s_nrt.20231119_230000_TAI.hdr",
  MagRadius:481.69385296937,
  FileIco:"Ico160/20231119_2258_Mix171_193_160px.jpg"}
 ],
 Channel:{name:"CoroHole",code:"CH",url:"/Hg/ch",
  opStep:"3h",chanIcoUrl:"/Hg/ch/ch_icon24_v01.png",
  maxSize:"1024",
  dtFrom:"2022-01-01",dtTo:"2023-11-19"
 },
 Times:{otherDays:null,
  otherMons:[
   "2023-01","2023-02","2023-03",
   "2023-04","2023-05","2023-06",
   "2023-07","2023-08","2023-09",
   "2023-10","2023-11"
  ],
  otherYears:null
 }
}

DayIco is sprite grid one jpeg for whole day... ix,iy,iw,ih specify position of this tile on the grid... ix and iy default to 0, it is sometimes missing in older json files...
(the sprite grid uses optimal column and row count depending on count of tiles needed, so for 8 images it will be 4x2 grid, for 24 images it will be 6x4 grid, for 25 images it will be 5x5 grid... it does not go too far from sqrt(count) to find the real optimum, so last one or more tiles may be blank black... for 26 images it would not use 13x2 but probably 6x5 with 4 blank tiles, or maybe 9x3 with 1 blank tile?) 

PAngle,Cx,Cy are for magnetogram from that time, not valid for CH jpeg... (I'm not sure why some items have it?? -- the item after 23:00 in latest day doesn't have interpolated position from SdoPos but the latest item... PAngle,Cx,Cy,RadAng,Source are normally removed during processing...) 
P.A.Semi #3
/Hg/ch/listing/20231101.json format:

All times are UTC...
Using Simplified JSON format - names are not quoted unless necessary...
Blue hyper-links in this description are for illustrative purpose:

{ dtFrom: "2023-11-01", dtTo: "2023-11-01",
 Items: [
  { Time: "2023-11-01T00:07:41",
    File: "20231101_000741_0193_HEq_HCw_1024px.jpg",
    Width: 1024, Height: 1024,
    Lon: 311.22088498075, Lat: 4.436454638495, Car: 2277,
    Dist: 213.36080992871,
    MagRadius: 479.50175241112,
    DayIco: "DayIco160/20231101_HCw_DayIco160.jpg",
    ix: 0, iy: 0, iw: 160, ih: 160,
    FileIco: "Ico160/20231101_000741_2048_0193_HEq_HCw_160px.jpg",
    FileSize: 327233, Modified: "2023-11-07T10:45:50"
  },
...
 ],
 Channel: { name: "HCw", code: "HCw", url: "/Hg/hcw",
  opStep: "3h", chanIcoUrl: "/Hg/hcw/hcw_icon24_v01.png",
  maxSize: "1024", dtFrom: "2022-01-01", dtTo: "2023-11-21"
 },
 Zip: "20231101_HCw.zip",
 Times: {
  otherDays: [ "2023-11-01", "2023-11-02", "2023-11-03",
   "2023-11-04", "2023-11-05", "2023-11-06", "2023-11-07",
   "2023-11-08", "2023-11-09", "2023-11-10", "2023-11-11",
   "2023-11-12", "2023-11-13", "2023-11-14", "2023-11-15",
   "2023-11-16", "2023-11-17", "2023-11-18", "2023-11-19",
   "2023-11-20", "2023-11-21"
  ],
  otherMons: [ "2023-01", "2023-02", "2023-03", "2023-04",
   "2023-05", "2023-06", "2023-07", "2023-08", "2023-09",
   "2023-10", "2023-11"
  ],
  otherYears: [ "2022", "2023" ]
 }
}

P.A.Semi #4
MonDaily lists one image per day, all (tiles 160x160) on same monthly sprite grid jpeg :

https://heliograf.org/Hg/hmib/202307_HMIB_MonDaily.json
/Hg/hmib/202307_HMIB_MonDaily.json format:

All times are UTC, and if there is 00:00:00 it is outright missing... "Z" at the end of time values is implied, and space can be converted to T for standard date parsing, so "2023-07-01" stands for "2023-07-01T00:00:00Z" and "2023-07-01 01:00:00" for "2023-07-01T01:00:00Z" ...

{dtFrom:"2023-07-01", dtTo:"2023-07-31",
 Items:[
  { Time:"2023-07-01",
    File:"hmi.M_720s.20230701_000000_TAI.3.magnetogram_1024px.jpg",
    SrcFile:"hmi.M_720s.20230701_000000_TAI.3.magnetogram.jpg",
    Width:1024,Height:1024,
    Lon:136.23998315925,Lat:2.8189368872601,Car:2272,
    Dist:218.48735558032,
    MagRadius:468.24243821821
  },
  ...
 ],
 ],
 DayGrids:[
  {dtFrom:"2023-07-01",dtTo:"2023-07-31", TileSize:160,
   TileFile:"202307_HMIB_MonIco160.jpg",
   TFSize:310526,
   Items:[
    { Time:"2023-07-01",
      File:"hmi.M_720s.20230701_000000_TAI.3.magnetogram_1024px.jpg",
      ix:0,iy:0,iw:160,ih:160
    },
    ...
    { Time:"2023-07-31",
      File:"hmi.M_720s.20230731_000000_TAI.3.magnetogram_1024px.jpg",
      ix:960,iy:480,iw:160,ih:160
    }
   ]
  },{dtFrom:"2023-07-01",dtTo:"2023-07-31",TileSize:80,
   TileFile:"202307_HMIB_MonIco80.jpg",TFSize:85866,
   Items:[
    { Time:"2023-07-01",
      File:"hmi.M_720s.20230701_000000_TAI.3.magnetogram_1024px.jpg",
      ix:0,iy:0,iw:80,ih:80
    },
    ...
    { Time:"2023-07-31",
      File:"hmi.M_720s.20230731_000000_TAI.3.magnetogram_1024px.jpg",
      ix:480,iy:240,iw:80,ih:80
    }
   ]
  }
 ],
 Channel:{name:"HMIB",code:"Mag",url:"/Hg/hmib",
  opStep:"3h",chanIcoUrl:"/Hg/hmib/hmib_icon24.png",
  maxSize:"1024",
  dtFrom:"2023-01-01",dtTo:"2023-11-21"
 },
 Zip:"202307_Hmib_MonDaily.zip",
 Times:{
  otherMons:[ "2023-01","2023-02","2023-03",
   "2023-04","2023-05","2023-06",
   "2023-07","2023-08","2023-09",
   "2023-10","2023-11"
  ],
  otherYears:["2023"]
 }
}

In case of HMIB channel, the newer files are from Jpegs available near-real-time (yesterday images), while Fits files are processed monthly...

So in same channel, the format of jpeg file-names may differ from month to month...

Link 202311_HMIB_MonDaily.json (in present non-complete month) redirects to  202311_Hmib_MonDaily_v231122_0126.json (which will not be there when you read it later...) :

{ dtFrom:"2023-11-01", dtTo:"2023-11-21",
 Items:[
  {Time:"2023-11-01",
   File:"20231101_000000_HMIBC_1024px.jpg",
   SrcFile:"20231101_000000_1024_HMIBC.jpg",
   Width:1024,Height:1024,
   Lon:311.29137069119,Lat:4.4373433654643,Car:2277,
   Dist:213.35964366925,
   MagRadius:479.5043622127
  },
...


Due to known inconsistency in jpeg file names, the zip operator selects proper image based on time and requested size (_1024px.jpg or _512px.jpg or _160px.jpg or _80px.jpg) within daily zip-file of that Channel regardless of the actual requested file name, but the file-name must match some of the patterns for that channel...

The actual response file name is in X-FileName: (or x-filename:) http response header...

...
P.A.Semi #5
Blog format - not sure yet, if it will be such publicly available, but there is not much more than what is also in blog html page:

/Hg/blog/api -- this article (html front-end)...
/Hg/blog/api.json -- storage of this article
/Hg/blog/api.comments - comments of this article

Internally, the path to store json & comments files is configurable and doesn't need to be simply same */blog/...

(the private information of comment authors is stored elsewhere, accessed by Handle, which is also in comments in blog html, which can be used to verify same user, from same address, posting that... otherwise the two hashes bear no content of those informations, which are not reversely accessible from the hashes... in blog, a user email serves as everywhere else as a password to detect fraud of other user posting in his name...) 

article.json format:

{id:null,
 articleId:"4493cee4f8d1",typ:2,
 slug:"api",
 title:"Heliograf API",
 created:1700675110.725,updated:1700693361,
 published:1700675161.599,
 author:{title:"P.A.Semi"},
 category:"Technical",
 categories:["Technical"],
 tags:["api"],
 options:{},hash:"80116cebdc4b363a483bbeb66a9ae843",
 idRevision:"3f10c03e584d",
 oldRevision:"abdf9e3c5c7d",
 Text:"<h1>Heliograf API<br></h1><p><br></p><p>...
 <font color=\"gray\">(will be described later)</font> ...
 <br></p><p><br></p>"
}

Time values in seconds since 1970 (unix/php time), where there are decimals, the time was assigned by Javascript client of the authoring tool (ECMA time / 1000) ... (the clocks are not necessarily in sync...) usually from "now" keyword...


article.comments format:

{ articleId: "4493cee4f8d1",
  created: 1700681892, updated: 1700694731,
  newSerial: 6,
 Items: [
...
  { id: null, aid: "4493cee4f8d1", typ: 12,
    cid: "fe47d58d896441fb", time: 1700697715.257,
    created: 1700697715.257, updated: 1700694731,
    ser: 5,
    author: { name: "P.A.Semi", handle: "7ba4997841fe.dd2f6443" },
    contents: "Blog format - not sure yet, if it will be such publicly available, but there is not much more than what is also in blog html page:<br><br><a target=\"_blank\" href=\"/Hg/blog/api\">/Hg/blog/api</a>  -- this article<br><a target=\"_blank\" href=\"/Hg/blog/api.json\">/Hg/blog/api.json</a>  -- storage of this article<br><a target=\"_blank\" href=\"/Hg/blog/api.comments\">/Hg/blog/api.comments</a>  - comments of this article<br><br>...."
  }
 ]
}


P.A.Semi #6
Currently there are these channels available:
  • ch - CoroHole from channels 193,171 ... 




  • hcw - High-contrast white 193 ...




  • pmheq - AIA Planckian Mapping from channels 171,211,304 ... 




  • pmch2 - Aia/PM with CH combined... 




  • hmii - HMI Intensitygrams (visible light) ... 




  • hmib - HMI Magnetograms ... 




  • lasco - SOHO Lasco C3 ... 




P.A.Semi #7
/Hg/chanRanges.json has listing of channels and available date ranges:

Presently the format is following:

{
 HMII:{name:"HMII",code:"Hmii",ymdFrom:"20230101",ymdTo:"20231122",
       url:"/Hg/hmii/"},
 HMIB:{name:"HMIB",code:"HMIB",ymdFrom:"20230101",ymdTo:"20231122",
       url:"/Hg/hmib/"},
 HCw:{name:"HCw",code:"HCw",ymdFrom:"20221231",ymdTo:"20231122",url:...},
 CoroHole:{name:"CoroHole",code:"CH",ymdFrom:"20221231",ymdTo:"20231119",url},
 PmHeq:{name:"PmHeq",code:"Pm",ymdFrom:"20230101",ymdTo:"20231114",url:...},
 PmCh2:{name:"PmCh2",code:"P-C",ymdFrom:"20230101",ymdTo:"20231114",url:...},
 Lasco:{name:"Lasco",code:"Lasco",ymdFrom:"20230101",ymdTo:"20231122",url:...}
}

The ymdTo value in yyyymmdd format can be used to quickly test for new data days in those channels...

Channel url cannot be simply derived from name and code, since /Hg/ch/ uses lowercase code, while /Hg/pmch2/ uses lowercase name ... (code is short text for gui button...) 
^ v