Take AWS ec2 volume instance snapshot

This post is just related to how can you take snapshot/backup of your AWS  ec2 instance's volume using boto.

Two method is their:- (which we can use)
1. Cron Method
2. Check time difference between last snapshot creation time and current time.

1. Cron method:- Two cron method (C1 ,C2 two cron)
                    i)C1 runs every hour, 5th minute
                       Picks up the current hour, say CHOUR
                       Pick up volumes defined with backup in hours(2_hour,4_hour ... etc) 
                       Take 2,4 as frequency accordingly
                       if(CHOUR%freq)==0 Take snapshot
                     ii)C2 runs every day, 12th hour
                       Pick up the current day, say CDAY
                       Pick up volumes defined with backup in days(1_day,7_day ... etc)
                       Take 1,7 as frequency accordingly
                       if(CDAY%freq)==0 Take snapshot

Pseudo Code:-   
       i) Make connection with aws.
       
          connection = boto.ec2.connect_to_region('Region_Name',                                aws_access_key_id='aws_access_key_id',aws_secret_access_key='aws_secret_access_key')

      ii)  a)   Take command line argument which type cron is this (backup_type = "day/hour")
            b)  Get all machine/instance which has same backup type as you passed in command line argument. 
            c)  Take snapshot those instances.  It's simple :)




2.   Check time difference between last snapshot creation time and current time method:-
      One cron can take back up for any type of back up freq.
       
      i) We have to check first that machine has any already any back up/ snapshot or not. If it has not simple take sanpshot that machine.
         other wise get all snapshot of that machine and sort them according to time creation. 
       eg:- 
           snapshots = conn.get_all_snapshots(filters={'volume_id':volume.id})//filter via volume id. or                                                                  //can also check boto documentation 
         
           snap_sort = sorted([(s.id,s.start_time) for s in snapshots] , key =lambda k :k[1])
         
           index = len(snap_sorted) - 1 // get the index of latest create snapshot
         
          snap = snap_sorted[index]   //snapshot value.
          
      Now just check time difference between last create snapshot and current time.
       eg:- 
             snapshotTimeDiff = (datetime.utcnow() - datetime.strptime(snap[1], "%Y-%m-%dT%H:%M:%S.000Z")).total_seconds()
   
      Now compare this with  backup freq for that instance.
             if  snapshotTimeDiff  >=  freqOfCreatingSnapshotInSecond:
                   (Take snapshot of this instance volume.)



Note:- 
       we have a machine config file where we mention every thing about ec2 instance.
       e.g:-  machine_configure.py
                "Instance_Name" : {
                    "backup_freq":"4_hour", "retention": "30_day",
                   "Environment" : "Dev",
                    "monitoring" : "basic",
                   "volume_override" : { // machine can have multiple volume and we want some volume back up freq diff from default so we choose this override 
      "volumne_id" : { "backup_freq":"4_hour", "retention": "7_day"},
      "volume_id" : { "backup_freq":"4_hour", "retention": "2_day"},
      "volume_id" : { "backup_freq":"1_day", "retention": "2_day"},
               }



Happy Coding :) 
Feel free to ask me to anything . It's My first blog.

Link :- 
       
            
           
   




Comments

Post a Comment

Popular posts from this blog

Enable logging in Haproxy.

HAProxy Log Setup HAProxy wants to log.   First HAProxy does not log directly into a file due to performance reason. So we need to handle  that with syslog server. But Haproxy also requires a syslog to listen on UDP port ( which in default syslog/rsyslog installation is not enabled ).   Basically log enable format in haproxy cfg is  “log <address> <facility> [max level [min level]]” Note:-  Adds a global syslog server. Up to two global servers can be defined. They will receive logs for startups and exits, as well as all logs from proxies configured        with "log global”.    <address> can be one of:        - An IPv4 address optionally followed by a colon and a UDP port. If no port is specified, 514 is used by default (the standard syslog port).        - A filesystem path to a UNIX domain socket, keepi...
Read more

Kruskal Algorithm.

//No one born perfect :) //Kruskal Algorithm //Finding MST (Minimum Spanning Tree) //Greedy Algo //O(ElogE or ElogV) //DS use DisJoint DS (Union Find)(very important DS) //How it is work ? (We are considering that Graph is connected ) // a). First Sort all edge in increasing order according to weight // b). Now Iterate each edge(u,v) in Edges and check whether these // vertax u and v are in same tree or not (which we are finding using union and find method) // if not include this edge otherwise continue (For more understanding see source code) //  c). You will get a MST in end. //For proof go wiki page:- // https://en.wikipedia.org/wiki/Kruskal%27s_algorithm#Proof_of_correctness // Happy Coding. ///////////////////////////////////////////////////////////////////////////////////////////// #include <iostream> #include <cstdio> #include <list> #include <cstring> using namespace std; // vector<vector <long int,long int...
Read more

Edit Distance Problem

/* ==========  ========== ========== ========= ======== ===========*/ //            CF - Edit Distance Problem                                                                               // //       Solution Code using  DP                                     // //            Author - Piyush Jain                                                                                           ...
Read more