troubleshooting Archives - Mind The Virtualization

Revolutionizing Kubernetes: Introducing Kubehelper, Your AI Sidekick!

Are you tired of the endless Google searches, scouring through pages just to find that elusive Kubernetes AI solution buried in the depths of the internet?

Have you ever wished for a reliable companion in your tech journey, someone who understands Kubernetes like the back of their hand? Well, your wish just came true! We are thrilled to introduce Kubehelper – your ultimate tech sidekick who’s here to revolutionize your Kubernetes experience.

Unraveling the Magic of Kubehelper

Kubehelper is not just another tool; it’s a game-changer. We’ve been burning the midnight oil, crafting a solution that simplifies your Kubernetes challenges, and putting an end to those frustrating forum hunts and late-night coding marathons. So, what sets Kubehelper apart from the rest? Let’s dive right in:

Corrupt Snapshot Causing View Composer Service Crash

Error during provisioning:Unexpected VC fault from View Composer

The above error might show up in Horizon View, if you try to use a corrupt snapshot for the deployment of new VMs.

How To Identify If A Corrupt Snapshot Is The Cause?

Look at the Microsoft System Event Log on your View Composer
- You should see something like this:
  
  The VMware View Composer service terminated unexpectedly. M It has done this 2 time(s). The following corrective action will be taken in 60000 milliseconds: Restart the service.
Search for FATAL messages in the vmware-viewcomposer.log
- The following message will indicate a corrupt snapshot
  
  2015-10-06 17:52:53,956 | WFE thread 9 | FATAL | ServiceCore.WorkflowEngine.WorkflowEngine – Unexpected exception occurred.Error reading single property ‘config.hardware.device’ from managed object: snapshot-15587

Continue reading “Corrupt Snapshot Causing View Composer Service Crash”

Remove VIB – Device or resource busy

Today, I was playing around with some vSphere Installation Bundles (VIB) and ran into an issue when I tried to remove vib:

esxcli software vib remove -n vmware-esx-KoraPlugin 
 [InstallationError]
 Error in running rm /tardisks/:
 Return code: 1
 Output: rm: can't remove '/tardisks/': Device or resource busy

Even adding the –force attribute did not help in this situation.

The following workaround seemed to be working for me:

Stophostd on theESXi host – this will be non-disruptive to yourVMs
```
/etc/init.d/hostd stop
```
Runlocalcli to uninstallVIB
```
localcli software vib remove -n vmware-esx-KoraPlugin
```
Note: We need to run localcli, since esxcli is not available if hostd is stopped
Starthostd on theESXi host
```
/etc/init.d/hostd start
```

Verify thatvmware-esx-KoraPlugin no longer shows up

esxcli software vib list | grep  vmware-esx-KoraPlugin

You should no longer see the VIB installed on your ESXi host.
Localcli is not widely spread within the community and mainly used by VMware’s Technical Support. It provides more troubleshooting capabilities, even if hostd is not running.

Silicon Valley OpenStack Ops Meetup

Yesterday, I attended the Silicon Valley OpenStack Ops Meetup and held a troubleshooting session focusing on Cinder, Keystone and Nova. The event was hosted by Nimble Storage, ElasticBox and SwiftStack. The focus of this Meetup was to share tips and tricks.

The event was hosted at Nimble Storage’s campus in San Jose, CA.

Even though the San Francisco Giants were playing their first World Series game, roughly 100 people attended the event. I think this is a pretty good turnout for the first Silicon Valley OpenStack Ops Meetup.

I was lucky and gathered a slot together with Wen Yu to cover OpenStack Shared Storage and Troubleshooting Tips and Tricks. To be honest, I have never been this nervous before. This was the first time speaking in front of more than 20 people.

Agenda:

5:45 PM – Doors Open, Food Served, Meet and Greet
6:20 PM – Bill Borsari & Pat Darisme ( Meetup Organizers ), Nimble Storage – Meet Up kickoff
6:30 PM – Ravi Srivatsav ( CEO ), ElasticBox – Avoiding cloud lock-in to give you total freedom to build, manage, and deploy applications faster than ever before.
6:50 PM – John Dickinson ( SwiftStack technical lead & OpenStack Swift PLM ), SwiftStack – Swift Product Line Manager talks about Object Storage and Swift in the Enterprise
7:10 PM – Wen Yu ( Nimble Product Manager ) & Jan Schwoebel ( Nimble Virtualization Support Lead ), Nimble Storage – OpenStack Shared Storage and TroubleShooting Tips and Tricks
7:30 PM – 9 PM – Meet the Presenters

Bill Bosari and Pat Darisme kicked off the event and welcomed all participants, who made it to the event even though the SF Giants had the first World Series Game.
Robin
- OverviewofElasticBox
  - Mission: ElasticBox empowers business to innovate faster by making it insanely easy for IT, ops and developers to build, manage and deploy applications in the cloud
  - Architecture:
    - Build any application and host it within any, supported, cloud (Amazon, Google, VMware, OpenStack,…)
    - Seamlessly migrate applications from cloud to cloud, don’t be locked down to one cloud solution
    - Share applications and “boxes” with people
      - Boxes are a bundle of packages

John Dickinson – Slides can be found here
- What is Swift?
  - Swift is an Object Store
  - Great for unstructured data which grows and grows (Images, Videos, Documents,…)
- What problem does Swift solve?
  - It is build for availability and durability
  - Users do no longer have to worry about where the data is located
  - Great manageability
  - Migrate data without any downtime for your users
- HowdoesSwiftStack fit in?
  - Provides a manage and control center for Swift
  - Add two additional components, controller & gateway
  - Gateway is a SMB/CIFS and NFS server
  - SwiftStack will provide an all-day workshop in SF on October 28th. Details can be found here

Wen Yu
- Value of Shared Storage
- Nimble Cinder Features
- ITO – Image Transfer Optimization.

Jan Schwoebel
- OpenStack Troubleshooting and Tips
- About me
- Troubleshooting Keystone
- Troubleshooting Cinder
- Troubleshooting Nova

Unfortunately, I haven’t received the slides from Robin and John yet. However, as soon as I receive them, I’ll add them to this post.

Crucial Data In Your VMware ESXi 5 Log Files

As an Escalation Engineer, part of my daily work is reviewing log files of various systems and vendors. In my first blog post, I would like to show which VMware ESXi 5 log files are most relevant for troubleshooting storage and networking related problems.

All current ESXi 5 logs are located under /var/log and as they rotate, they’ll be available under /scratch/logs

/var/log/vmkernel.log:

VMkernel related activities, such as:
- Rescan and unmount of storage devices and datastores
- Discovery of new storage like iSCSI and FCP LUNs
- Networking (vmnic and vmks connectivity)

/var/log/vmkwarning.log:

Extracted warning and alert messages from the vmkernel.log

/var/log/hostd.log::

Logs related to the host management service
SDK connections
vCenter tasks and events
Connectivity to vpxa service, which is the vCenter agent on the ESXi server

/var/log/vobd.log:

VMkernel observations
Useful for network and performance issues

Also, if you have a VM which is affected in particular, it might be worth looking into the vmware.log which is stored with the Virtual Machine. You can find the log under /vmfs/volumes/datastore_name/VM_name/vmware.log.

For the location of ESXi 3.5 and 4.x log files, can be found here.