Introduction to Python Boto3

Cloud computing is a type of Internet-based computing that provides shared computer processing resources and data to computers and other devices on demand.

The goal of cloud computing is to allow users to take benefit from all of these technologies, without the need for deep knowledge about or expertise with each one of the

Different Cloud services:

 

AWS Basics and terminology:.

Following are some basic concept and terminology on AWS.

AWS provides a variety of computing and networking services to meet the needs of your applications. You can provision virtual servers, set up a firewall, configure Internet access, allocate and route IP addresses, and scale your infrastructure to meet increasing demand.

Some Basic AWS services:

Amazon EC2: It provides virtual servers in the AWS cloud.

Amazon VPC:  It provides an isolated virtual network for your virtual servers.

Elastic Load Balancing: Distributes network traffic across your set of virtual servers.

Auto Scaling:  Automatically scales your set of virtual servers based on changes in demand.

Amazon Route   53: Routes traffic to your domain name to a resource, such as a virtual server or a load balancer.

AWS Lambda: Runs your code on virtual servers from Amazon EC2 in response to events.

Amazon ECS: Provides Docker containers on virtual servers from Amazon EC2.

Basic AWS terminology:

An Amazon Machine Image (AMI) is a template that contains a software configuration (for example, an operating system, an application server, and applications). From an AMI, you launch an instance, which is a copy of the AMI running as a virtual server on a host computer in Amazon’s data center.

VPCs and Subnets: A virtual private cloud (VPC) is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS cloud, providing security and robust networking functionality for compute resources.

A subnet is a segment of a VPC’s IP address range that you can launch instances into. Subnets enable you to group instances based on your security and operational needs.

A security group acts as a virtual firewall for your instance to control inbound and outbound traffic.

Introduce to Boto3:

What is Boto?

AWS SDK for Python Developers.

Interact AWS easily through Python.

As of Now it supports more than 35 Services.

Boto 3:

Installation:

Install the latest Boto 3 release via pip:

`pip install boto3`

Configuration:

Before you start using Boto 3, you should set up authentication credentials.

If you have the `aws cli` installed, then you can use it to configure your credentials file:

`aws configure`

Or else you can create the credential file and kept it at:

~/.aws/credentials:

[default]

aws_access_key_id = YOUR_ACCESS_KEY

aws_secret_access_key = YOUR_SECRET_KEY

You may also want to set a default region.  and kept at ~/.aws/config:

[default]

region= us-west-2a

Using Boto3: 

On this demonstration, I will explain about to create, use and delete following resources on Amazon EC2.

1.Create a t2 instance:

ec2=boto3.resource (‘ec2′)

instance = ec2.create_instances(

ImageId=’ami-b04e92d0′,

MinCount=1,

MaxCount=1,

KeyName=’new key’,

InstanceType = ‘t2.micro’,

Placement = {

‘AvailabilityZone’: ‘us-west-2b’,

},

SecurityGroupIds=[

‘sg-9dd2b5e4’

],

SubnetId = ‘subnet-2f137c4b’,

)

ec2_client=boto3.client(‘ec2’)

waiter=ec2_client.get_waiter(‘instance_running’)

waiter.wait(InstanceIds=[instance[0].instance_id])

Note: [Waiter function will wait till the instance state change to “running”]

2.Create a EBS volume:

volume = ec2.create_volume(

Size=32,

AvailabilityZone=’us-west-2b’,

VolumeType=’standard’,

)
waiter = ec2_client.get_waiter(‘volume_available’)

waiter.wait(VolumeIds=[volume.volume_id])

3.Attach this Volume to the instace

response = volume.attach_to_instance(

InstanceId=instance[0].instance_id,

Device=’sdb’)

waiter = ec2_client.get_waiter(‘volume_in_use’)

waiter.wait(VolumeIds=[volume.volume_id])

4.create an elastic IP and associate address:

elastic_ip=ec2_client.allocate_address(Domain=’VPC’)

response = ec2_client.associate_address(

InstanceId=instance[0].instance_id,

PublicIp=elastic_ip[‘PublicIp’],

)

5. Detach EBS from the instance

ec2_client.detach_volume(

VolumeId=volume.volume_id,

Device=’sdb’,

InstanceId=instance[0].instance_id,

Force=True

)

waiter = ec2_client.get_waiter(‘volume_available’)

waiter.wait(VolumeIds=[volume.volume_id])

6.Delete EBS volume
ec2_client.delete_volume(VolumeId=root_vol_id)

7.Terminate the instance

response=instance[0].terminate()

instance[0].wait_until_terminated()

8. Release the public IP.

ec2_client.release_address(AllocationId = elastic_ip[‘AllocationId’])

 

SAN QA

1Q: What is the difference b/w SAN and NAS?

The basic difference between SAN and NAS,
SAN – Storage Area Network, SAN is Fabric based.I
It accesses data on block level and produces space to host in form of disk.It               uses block access protocol to access data(i.e iscsi,FC,FCOE,FCIP etc)

NAS – Network attached Storage, NAS is Ethernet based.
It uses file access protocol to access data(i.e NFS,CIFS etc)
It accesses data on file level and produces space to host in form of shared network folder.

2Q: What are the different typologies in Fibre Channel?

a)Point-to-Point
b)Arbitrary Loop
c)Switched Fabric Loop

3Q: What are the layers of Fibre Channel Protocol?

Ans: a)FC Physical Media
b)FC Encoder and Decoder
c)FC Framing and Flow control
d)FC Common Services
e)FC Upper Level Protocol Mapping

4Q: What is zoning?

Ans: Fabric management service that can be used to create logical subsets of devices within a SAN. This enables portioning of resources for management and access control purpose.

Q5: What are the two major classification of zoning?

Ans: Two types of zoning are
a) Software Zoning
b) Hardware Zoning

Q6: What are different levels of zoning?

Ans: a)Port Level zoning
b)WWN Level zoning
c)Device Level zoning
d)Protocol Level zoning
e)LUN Level zoning

Q7: How does FC Switch maintain the addresses?

Ans: FC Switch uses simple name server(SNS) to maintain the mapping table.

Q8: What is Multi-path I/O?

Ans: Fault tolerant technique where, there is more than one physical path between the CPU in the computer systems and its main storage devices through the buses, controllers, switches and other bridge devices connecting them.

Q9: What is the smallest unit of information transfer in FC?

Ans: Frame

Q10: What is a sequence in FC?

Ans: Group of one or more frames that encompasses one or more “information units” of a upper layer protocol.

Example:
It requires
i) One sequence to transfer the command
ii) One or more sequence to transfer the data
iii) Once sequence to transfer the status.

Q11: What is Exchange in FC?

Ans: Exchange is to establish a relationship between 2 N_PORTs and then these two ports transfer data via one or more sequence within this relationship.
Example: Exchange exist to transfer the command, data and the status of one SCSI task

Q12: What are the different types of FC logins and its purpose ?

Port Login: To exchange service parameters between N_Ports and N_Ports
Process Login: To establish the SCSI operating environment between two N_PORTS
Fabric Login: Similar to port login, FLOGI is an extended link service command that sets up a session between two participants. With FLOGU a session is created between an N_Port or NL_Port and the switch.

Q13: What is WWPN?

Ans: WWPN is the 16bit character that is assigned to the port, SAN volume controller uses it to uniquely identify the fibre channel HBA that is installed in the host system.

Q14: What is the highest and lowest priority of SCSI?

There are 16 different ID’s which can be assigned to SCSI device 7, 6, 5, 4, 3, 2, 1, 0, 15, 14, 13, 12, 11, 10, 9, 8.

Highest priority of SCSI is ID 7 and lowest ID is 8.

Q15: Which one is the Default ID for SCSI HBA?

Generally the default ID for SCSI HBA is 7.

Q16 :What are different types of protocols used in transportation and session layers of SAN?

Fibre Channel Protocol (FCP).

Internet SCSI (iSCSI).

Fibre Channel IP (FCIP).

Q17: What are the protocols used in physical/datalink and network layer of SAN?

Ethernet.

SCSI.

Fibre Channel.

Q18: Describe in brief the composition of FC Frame?

Start of the Frame locator

Frame header (includes destination id and source id, 24 bytes/6 words).
Data Payload (encapsulate SCSI instruction can be 0-2112 bytes in length).
CRC (error checking, 4 bytes).
End of Frame (1 byte).

 

 

 

 

 

 

Linux Essentials For QA

Q1:Explain in detail the Linux booting procedure
Ans:
• When a Linux machine is powered on BIOS loads up first. It will prompt you to select boot device which can be Hard disk, CD-ROM, Floppy drive, Network etc. By default generally it will boot with hard disk
• Next comes your MBR. This will load and execute the GRUB boot loader menu. GRUB stands for Grand Unified Boot Loader. This will display the a splash screen with the contents of /boot/grub/grub.conf
• List of available and installed kernels will be shown, if not selected default kernel will be loaded
• Kernel: Mounts the root files system as specified by “root=” parameter inside /boot/grub/grub.conf file
• Next it will execute /sbin/init program which will boot the linux machine in the default run level as specified by /etc/inittab
• Runlevel: All the scripts loaded inside the selected runlevel from step 5 will be executed These scripts are placed inside /etc/rc.d/rcx.d/. Here x is the runlevel value which will be varying from 0-6
• Scripts starting from S would load at startup and those starting with K would kill the process at shutdown. These incident will take place as per the numerical value assigned to them. For eg: s13network will load prior than s15 sendmail
• Next your login screen will come up.

Q2: What are the 6 run levels of linux? And how can you configure your script to run only when the system boots into GUI and not to any other runlevel
Ans:      0 power off
1 single user
2 multi user without network
3 multiuser with network
4 development purpose
5 GUI
6 Restart

chkconfig –level 5 service_name on
chkconfig –level 1234 service_name off

Q3:What is the difference between soft link and hard link?
Ans:
Soft Link
Using this only a link to the original file is created (shortcut).
It contains path to the original file and it have different inode no.
If you delete the file then the created link (shortcut) won’t work.
In case you delete the shortcut link then it won’t affect the original file

Hard Link
Another copy of the file is created.
Both the file have same inode no.
Any changes made in either of the file will appear on the other file.
Deleting any of the one file won’t affect the other file.

Q4: What is inode ? Briefly explain the structure ?
Ans: Inode is a data structure that keeps track of all the information about a file. When we keep our information in a file and the OS stores the information about a file in an inode. Information about files is sometimes called metadata. We can say that an inode is metadata of the data. In a file system, inodes consist roughly of 1% of the total disk space, whether it is a whole storage unit or a partition on a storage unit. The inode space is used to ?track? the files stored on the hard disk. The inode entries store metadata about each file, directory or object, but only points to these structures rather than storing the data. Each entry is 128 bytes in size. The metadata contained about each structure can include the following:

Inode number
Access Control List (ACL)
Extended attribute
Direct/indirect disk blocks
Number of blocks
File access, change and modification time
File deletion time
File generation number
File size
File type
Group
Number of links
Owner
Permissions
Status flags

Inode structure of a directory consists of a name to inode mapping of files and directories in that directory.In a directory, You can find the inode number corresponding to the files using the command “ls -i”

Q5: How to find the bit size of your linux machine?
Ans: # uname -m
i686

# uname -m
x86_64

If you get i386, i586 and i686 in your reply than that signifies your machine is 32-bit but if you get x86_64 or ia64 then your machine will be 64-bit.

# getconf LONG_BIT
32

# getconf LONG_BIT
64

Here you will get an output of bit size either 32 or 64

Q6: How can you add a banner or login message in Linux?
Ans: By editing these two files
/etc/issue
/etc/motd

Q7: Which log file will you check for all authentication related messages?
Ans: /var/log/secure

Q8: What is the command to create multiple directories using one command?
Ans: Using -p argument along with mkdir command

Q9: Which command would you prefer to monitor your system performance and why?
Ans: I personally prefer top and sar command to view the performance related factors of my Linux machine.

The top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel.

Also sar gives us a standard o/p of the running activity performance related factors in relation to paging, NIC, CPU, Memory etc

Q10: What is an orphan process?
Ans:An orphan process is a process that is still executing, but whose parent has died.

Q11: What is a defunct process?
Ans:These are also termed as zombie process. These are those process who have completed their execution but still has an entry in the process table. When a process ends, all of the memory and resources associated with it are de-allocated so they can be used by other processes.After the zombie is removed, its process identifier (PID) and entry in the process table can then be reused.

Zombies can be identified in the output from the Unix ps command by the presence of a “Z” in the “STAT” column

Q12: Why LVM is required ?
Ans: LVM stands for Logical Volume Manager , to resize filesystem’s size online we required LVM partition in Linux. Size of LVM partition can be extended and reduced using the lvextend & lvreduce commands respectively.

Q13: How To check Memory stats and CPU stats ?
Ans: Using ‘free’ & ‘vmstat’ command we can display the physical and virtual memory statistics respectively.With the help of ‘sar’ command we see the CPU utilization & other stats.

Q14: What does Sar provides and at which location Sar logs are stored ?
Ans: Sar Collect, report, or save system activity information. The default version of the sar command (CPU utilization report) might be one of the first facilities the user runs to begin system activity investigation, because it monitors major system resources. If CPU utilization is near 100 percent (user + nice + system), the workload sampled is CPU-bound.
By default log files of Sar command is located at /var/log/sa/sadd file, where the dd parameter indicates the current day.

Q15: How to increase the size of LVM partition ?
Ans: Below are the Logical Steps :
– Use the lvextend command (lvextend -L +100M /dev/<Name of the LVM Partition> , in this example we are extending the size by 100MB.
– resize2fs /dev/<Name of the LVM Partition>
– check the size of partition using ‘df -h’ command

Q16: How to reduce or shrink the size of LVM partition ?
Ans: Below are the logical Steps to reduce size of LVM partition :
-Umount the filesystem using umount command,
-use resize2fs command , e.g resiz2fs /dev/mapper/myvg-mylv 10G
-Now use the lvreduce command , e.g lvreduce -L 10G /dev/mapper/myvg-mylv
Above Command will shrink the size & will make the filesystem size 10GB.

Q17: What is the command to display all the logical volume available in the system?
Ans:lvdisplay

Q18: How to create partition from the raw disk ?
Ans: Using fdisk utility we can create partitions from the raw disk.Below are the steps to create partition from the raw dsik :
– fdisk /dev/hd* (IDE) or /dev/sd* (SCSI)
– Type n to create a new partition
– After creating partition , type w command to write the changes to the partition table.

Q19: Where the kernel modules are located ?
Ans: The ‘/lib/modules/kernel-version/’ directory stores all kernel modules or compiled drivers in Linux operating system. Also with ‘lsmod’ command we can see all the installed kernel modules.

Q20: What is umask ?
Ans: umask stands for ‘User file creation mask’, which determines the settings of a mask that controls which file permissions are set for files and directories when they are created.

Q21: How to set the umask permanently for a user?
Ans: To set this value permanently for a user, it has to be put in the appropriate profile file which depends on the default shell of the user.

Q22: How to change the default run level in linux ?
Ans: To change the run level we have to edit the file “/etc/inittab” and change initdefault entry ( id:5:initdefault:). Using ‘init’ command we change the run level temporary like ‘init 3’ , this command will move the system in runlevl 3.

Q23: How to share a directory using nfs ?
Ans: To share a directory using nfs , first edit the configuration file ‘/etc/exports’ , add a entry like
‘/<directory-name> <ip or Network>(Options)’ and then restart the nfs service.

Q24: How to check and mount nfs share ?
Ans: Using ‘showmount’ command we can see what directories are shared via nfs e.g ‘showmount -e <ip address of nfs server>’.Using mount command we can mount the nfs share on linux machine.

Q25: List the services that are enabled at a particular run level in linux server ?
Ans: With the help of command ‘chkconfig –list | grep 5:on’ we can list all the service that are enabled in run level5. For other run levels just replace 5 with the respective run level.

Q26: How to enable a service at a particular run level ?
Ans: We can enable a service using the Command ‘chkconfig <Service-Name> on –level 3’

Q27: How to upgrade Kernel in Linux ?
Ans: We should never upgrade Linux Kernel , always install the new New kernel using rpm command because upgrading a kenel can make your linux box in a unbootable state.

Q28: How To scan newly assigned luns on Linux box without rebooting ?
Ans: There are two ways to scan newly assigned luns :
Method:1 if sg3 rpm is installed , then run the command ‘rescan-scsi-bus.sh’
Method:2 Run the Command , echo ” – – – ” > /sys/class/scsi_host/hostX/scan

(i.e for HOST in `ls /sys/class/scsi_host/`; do echo ‘- – -‘ > /sys/class/scsi_host/$HOST/scan; done) it will scan luns from all  port.

Q29: How to find WWN numbers of HBA cards in Linux Server ?
Ans: We can find the WWN numbers of HBA cards using the command ‘systool -c fc_host -v | grep port_name’

Q30: What is the command to view all the kernel parameters?
Ans: Sysctl -a

Q31: How to add & change the Kernel parameters ?
Ans: To Set the kernel parameters in linux , first edit the file ‘/etc/sysctl.conf’ after making the changes save the file and run the command ‘sysctl -p’ , this command will make the changes permanently without rebooting the machine.

Q32: What is load average in a linux ?
Ans: Load Average is defined as the average sum of the number of process waiting in the run queue and number of process currently executing over the period of 1,5 and 15 minutes. Using the ‘top’ and ‘uptime’ command we find the load average of a linux sever.

Q33: What is Kerberos used for?
Ans:Kerberos is used for secure network logon.

Q34: Which partition store system configuration files in Linux system?
/etc partition stores system configuration files in Linux.

Q35: What is the purpose of the swapon command ?
swapon command is used to activate a already created swap partition. It cannot be used to create a new swap partition.

Q36: Where are the automatically mounted file systems listed?
Ans:The automatically mounted file systems are listed in the file /etc/fstab.
Samba is used to provide support for which protocol?
Samba is used to support the Microsoft SMB/CIFS systems.

Q37: How to fix a case when a system is unable to boot due to improper entry in fstab?
Ans: Boot into single user mode and make the necessary corrections inside fstab

Q38: What is the default mode when you enter single user mode?
Ans: read only for the first time but once you change it to read write then from next time onward it will be read write

Q39: What command is used to load a module into the kernel?
Ans:The insmod command is used to load a module into the kernel

Q40: With respect to a Linux system’s RAM, how large should the swap file be?
Ans:Generally the size of the swap file should be twice the size of installed RAM.

Q41: What is Difference between .bashrc and .bash_profile
Ans:Every time you login to a Linux (Red Hat) machine .bash_profile file is executed
But , In case you are already logged in and you open a new terminal then .bashrc file is executed
So, basically you can put the environment variable inside any of the two files. As per me I would advice you to put the same inside .bash_profile.

Q42: What is . and .. in very directory of Linux system?
Ans: So basically single dot “.” means current directory which actually is a hardlink to its containing directory.
And double dot “..” means one step back i.e. the parent directory, that is the only directory where that directory is referenced from

Q43: What is the command used to check any hardware changes made on the system in Linux?
Ans: kudsu

Q44: Which command is used in linux to know the driver version of any hardware device?
Ans: dmesg

Linux Networking:

Q1: What are the default ports used for SMTP,DNS,FTP,DHCP,SSH and squid ?
Ans: Service Port
SMTP 25
DNS 53
FTP 20 (data transfer) , 21 ( Connection established)
DHCP 67/UDP(dhcp server) , 68/UDP(dhcp client)
SSH 22
Squid 3128

Q2: What is Network Bonding ?
Ans: Network bonding is the aggregation of multiple Lan cards into a single bonded interface to provide fault tolerance and high performance. Network bonding is also known as NIC Teaming.

Q3: What are the benifits of NIC Teaming?
Ans: Load balancing
Fault Tolerance
Failover

Q4: What are the different modes of Network bonding in Linux ?
Ans: Below are list of modes used in Network Bonding :
balance-rr or 0 – round-robin mode for fault tolerance and load balancing.
active-backup or 1 – Sets active-backup mode for fault tolerance.
balance-xor or 2 – Sets an XOR (exclusive-or) mode for fault tolerance and load balancing.
broadcast or 3 – Sets a broadcast mode for fault tolerance. All transmissions are sent on all slave interfaces.
802.3ad or 4 – Sets an IEEE 802.3ad dynamic link aggregation mode. Creates aggregation groups that share the same speed & duplex settings.
balance-tlb or 5 – Sets a Transmit Load Balancing (TLB) mode for fault tolerance & load balancing.
balance-alb or 6 – Sets an Active Load Balancing (ALB) mode for fault tolerance & load balancing.

Q5: How to check and verify the status the bond interface.
Ans: Using the command ‘cat /proc/net/bonding/bond0’ , we can check which mode is enabled and what lan cards are used in this bond. In this example we have one only one bond interface but we can have multiple bond interface like bond1,bond2 and so on.

Q6: How to check default route and routing table ?
Ans: Using the Commands ‘netstat -nr’ and ‘route -n’ we can see the default route and routing tables.

Q7: How to check which ports are listening in my Linux Server ?
Ans: Use the Command ‘netstat –listen’ and ‘lsof -i’

Q8: What is the command to check all the open ports of your machine?
Ans: nmap localhost

Q9: How can you make a service run automatically after boot?
Ans: using chkconfig command

Q10: Mention all the network configuration files you would check to configure your ethernet card
Ans: /etc/sysconfig/network-scripts/ifcfg-eth*
/etc/resolve.conf

Q11: What is the the use of /etc/resolv.conf?
Ans: It contains the details of nameserver i.e details of your DNS server which helps us connect to Internet

Q12. What is the use of /etc/hosts file?
Ans: To map any hostname to its relevant IP

Q13. What is a 3 way handshake protocol?
Ans: SYN – system 1 sends SYN signal to rmote system
SYN-ACK – remote sysstem receives the syn signal and sends ack signal
ACK – system again receives ack signal from remote system and connection is established

For Example: When you ping to a machine you are sending a SYN signal which is ACK by the remote machine then it sends a SYN ACK signal back to the host machine. Then the host machine receives SYN ACK and sends the ACK signal back to confirm the same.

Q14. What are the possible ways to check if your system is listening to port 67
Ans: nmap localhost | grep 67
netstat -ntlp | grep 67

How to create multiple iscsi sessions with Single target ip.

Creating Maximum iscsi sessions from a single RHEL server with following procedures.

1. Discover the iscsi target ips:
   iscsiadm -m discovery -t st -p <target ips>
   e.g iscsiadm -m discovery -t st -p 192.168.1.1

2. Open iscsi initiator (i.e iscsiadm) utility provides a feature to create multiple interface.
   iscsiadm -m iface -I <iface name> -o new 
   e.g iscsiadm -m iface -I iface.rhel-1 -o new

3. Now update the newly created interface with a uniq initiator name.
   iscsiadm -m iface -I <iface name> -o update -n iface.initiatorname -v <initiator name>
   e.g iscsiadm -m iface -I iface.rhel-1 -o update -n iface.initiatorname -v <iqn::name>

4. login to the target ip with this newly created interface 
   iscsiadm -m node --targetname <target iqn> -I <iface name> -p <target ip> --login
   e.g iscsiadm -m node --targetname iqn::name -I iface.rhel-1 -p 192.168.1.1 --login

With this above procedure we can create maximum no of interface and associate those interface  with uniq initiator name, which will provide multiple iscsi initiator on your RHEL server. 
With out using  multiple physical/virtual hosts we can create maximum sessions on RHEL host.

Journey on Python3 . . .

23rd Jan 2016 was a great day to start journey on Python3  with great mentors like  Kushal Das and Sayan Chowdhury  at Red Hat Pune.

Discussed interested logic on python3 like FizzBuzz game in generic as well as in list Comprehensive.

Note:FizzBuzz game is just to print nos  from  1 to 100, But if a no is divisible by 3 then it should print Fizz.if a no is divisible by 5 then it should print Buzz.if a no is divisible by both 3 and 5 then it should print FizzBuzz.

FizzBuzz

In Generic Way:
for i in range(1,100):
     if  i %3 ==0 and i%5 ==0 :
        print ("FizzBuzz", end=' ')
     elif i%5 ==0:
        print ("Buzz", end=' ')
     elif i%3 ==0:
        print ("Fizz", end=' ')
     else:
        print (i, end=' ')

In List Comprehensive:
['FizzBuzz' if i%3 ==0 and i%5==0 else 'Fizz' if i%3==0 else 'Buzz' if i%5 ==0 else i for i in range(0,101)]