Latest Post

Fluentd error: Unable to push logs to [elasticsearch]

-
After application deployments, Kibana stopped showing logs exactly after 7 days. The error "Fluentd error: Unable to push logs to [elasticsearch]" was shown in the fluentd logs. The initial response was to increase the buffer limits for fluentd as follows: chunk_limit_size 10M queue_limit_length 256 The behavior occurred again after two weeks, which led to the same error. On closer investigation, the error was preceded by the statement "Failed to write to the buffer." This led me to inspect the fluentd configuration again and found the following code in the buffer part which caused the fluentd buffers to be filled as per the official documentation on Fluentd : overflow_action block The fix for this overflow_action is to change from block to drop_oldest_chunk, allowing the fluentd logs to flow seamlessly to the elastic search by dropping the oldest logs in the buffer.   <buffer> @type file path /var/log/fluentd-buffers/kubernet
Post a Comment

Windows 7/2008 VM not booting up on Hyper-V host after windows update using CAU

-
After updating three hosts out of five in a windows hyper-v cluster based on Nutanix using CAU tool, I noticed the Windows 7/2008 VMs stopped running on the updated hosts. Using the failover cluster manager, I managed to failover the VM to the non-updated host and it started working.

During troubleshooting, I came across the below two blogs which set me thinking:



http://kx.cloudingenium.com/microsoft/servers/hyper-v/resolved-hyper-v-general-access-denied-error-trying-load-virtual-hard-drive/

https://blogs.technet.microsoft.com/rmilne/2013/10/22/windows-hyper-v-2012-8-1-hyper-v-did-not-find-virtual-machine-to-import/


In the Hyper-V manager, I noticed the Windows 2012 VMs were running without any issue but the Windows 7/2008 VMs were shown with the message  "Cannot connect to Virtual Machine storage". I was able to access the storage from the hosts without any problem and confirmed this with Nutanix support. 

The Hyper-V hosts create symbolic links to the XML files of the VMs hosted in "C:\Programdata\Microsoft\Windows\Hyper-V\Virtual Machines". The IDs did not match. 
I deleted the Symbolic links in the Hyper-V host and then failed over the same VMs using the Failover Cluster Manager to the updated host from the non-updated host and the VMs started to work.


Comments

Post a Comment

Popular posts from this blog

Office 365 User unable to book room on-premise in Exchange Hybrid environment

-
In a federated environment, an Office 365 user was unable to book meeting in room which was on-premise. After tracing the messages created for booking the room, it was confirmed that the message indeed reached but the room was rejecting it for some reason. The room was rejecting the meeting request as it was considering the Office 365 user as an external user. So to solve this, we need to run the following command in the On-premise Exchange environment if we want to allow all the rooms: Get-Mailbox | where {$_.RecipientTypeDetails -like "Roommailbox"}| Get-CalendarProcessing | Set-CalendarProcessing -ProcessExternalMeetingMessages $true For single room, run the following command: Get-CalendarProcessing -Identity <Room name> | Set-CalendarProcessing -ProcessExternalMeetingMessages $true  Now the Office 365 user will be able to book the on-premise meeting room. (This is assuming that the free/busy calendar sharing is already enabled in the federated environme
Read more

On-board Linux computers to Azure Log Analytics

-
For on-boarding linux servers to Azure log analytics, just execute the command on the respective server: wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -p [protocol://][user:password@]proxyhost[:port] -w <YOUR WORKSPACE ID> -s <YOUR WORKSPACE PRIMARY KEY> In few cases like mine, the servers will not have access to internet and we will need to install the downloaded oms agent package with the proxy information.  sh omsagent-1.6.0-42.universal.x64.sh --install -w <YOUR WORKSPACE ID> -s <YOUR WORKSPACE PRIMARY KEY> -p [protocol://][user:password@]proxyhost[:port] The proxy can have username and password for authentication and even if no proxy authentication is required, we will need to enter a dummy username and password which in the example below is azure and azure. sh omsagent-1.6.0-42.universal.x64.sh --install -w <YOUR WORKSPACE ID> -s
Read more

Fluentd error: Unable to push logs to [elasticsearch]

-
After application deployments, Kibana stopped showing logs exactly after 7 days. The error "Fluentd error: Unable to push logs to [elasticsearch]" was shown in the fluentd logs. The initial response was to increase the buffer limits for fluentd as follows: chunk_limit_size 10M queue_limit_length 256 The behavior occurred again after two weeks, which led to the same error. On closer investigation, the error was preceded by the statement "Failed to write to the buffer." This led me to inspect the fluentd configuration again and found the following code in the buffer part which caused the fluentd buffers to be filled as per the official documentation on Fluentd : overflow_action block The fix for this overflow_action is to change from block to drop_oldest_chunk, allowing the fluentd logs to flow seamlessly to the elastic search by dropping the oldest logs in the buffer.   <buffer> @type file path /var/log/fluentd-buffers/kubernet
Read more