using pscp.exe in a batch file and getting around the host key

I was trying to automate an scp file copy for work and having a hell of a time with the host key. Even though I was logged in as the same user that the script is running, when it’s run non interactive, for some reason it cannot find the host key in the registry, and fails. Some info.

• putty and  pscp.exe keep their host keys in – [HKEY_CURRENT_USERSoftwareSimonTathamPuTTYSshHostKeys]
• if you run your pscp.exe with the –batch command, you cannot input any console input
• in order to get this to work, I had to echo a “Y” to it like this – echo y | C:_scriptspscp -sftp –v  -etc. etc

Basically, If you try to echo the Y with the –batch option, then you can’t because the –batch option makes it ignore any input. In order to make this secure, I ran it once the way I have listed above, and once the host key was stored in the right place, I removed the echo and added the batch again.

Advertisement

strange .net runtime error and the fix

We had done a POC for Foglight by Quest back in March and ended up not purchasing the product because we already had SCOM in place. After the uninstall and ever since, we have had some issues.

• errors logged whenever an app pool starts/restarts containing this description

• .NET Runtime version 2.0.50727.3603 – Failed to CoCreate profiler.

.net health monitoring would log an app pool start but not log an app pool stop or shutdown

I had the suspicion that this was related to random app pool restarts during the day on one of our webservices

other debuggers would not attach to an app pool and work (like avicode that comes with SCOM)

I looked for how to fix this since March, of course I was too stubborn to call Quest, so that’s my fault. Much googling didn’t resolve much so when I finally did figure this out, I wanted to post what I found. It turns out that debugging is set in an environment variable. Seems like it’s a session variable because it’s set in the parameters of a service, so that way it runs with the service in the user context of the service. This is something I had never run across before so it seemed kinda odd.

In the end we had to remove the reg key “Environment” (and the contents of the key) from the two locations. This key is what sets the debugger to enabled and tells it which debugger to use.

• [HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesW3SVC]
• [HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesIISADMIN]

Once these have been deleted, all you have to do is IISRESET the server and the error is gone and random app pool restarts have ceased. As of this writing I have not tried to reinstall the SCOM AVIcode to the webservice, but I’m certain that it will work now.

AV exclusions

This comes up often and I can never find everything in one place. This is a good post with a bunch of info and links.

SQL query to tell if SQL logins are using Kerberos or NTLM

Not explanation needed here, if you know what you are looking for, this is it.

SELECT 
s.session_id
, c.connect_time
, s.login_time
, s.login_name
, c.protocol_type
, c.auth_scheme
, s.HOST_NAME
, s.program_name
FROM sys.dm_exec_sessions s
JOIN sys.dm_exec_connections c
ON s.session_id = c.session_id

MSMQ MP for SCOM–has issues..

This is just an example of the kind of stuff I find in the MSMQ MP for scom. Is this alert about the SIZE of the messages or the NUMBER of messages?

scom missing performance counters

I recently had to make some changes to my scom test environment server. It’s a single virtual machine with scom, sql, ssrs, the works on it. Despite what one would think, it has performed well with about 30 agents. Recently I added about that many more and it pushed up the ram to the 4gb max that I had given it. When I added more I realized I had used 2008 x32 for the OS (oops) and was not able to see the extra ram. After some research I found that I could do an upgrade from 2008 standard to enterprise, which is what I did. Afterwards, scom wouldn’t run because the SDK service would not start. This is the event log message.

Log Name:      Operations Manager
Source:        OpsMgr SDK Service
Date:          1/7/2011 2:25:48 PM
Event ID:      26380
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:     

Description:
The System Center Operations Manager SDK Service failed due to an unhandled exception. 
The service will attempt to restart.
Exception:

System.TypeInitializationException: The type initializer for ‘Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess’ threw an exception. —> System.InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.
   at System.Diagnostics.PerformanceCounter.Initialize()
   at System.Diagnostics.PerformanceCounter.set_RawValue(Int64 value)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager.CreateCounter(String categoryName, String counterName)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager.CreateCounters(PerformanceCounterCategoryDescriptor categoryInfo)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager..ctor(String instanceName, PerformanceCounterCategoryDescriptor categoryInfo)
   at Microsoft.EnterpriseManagement.Mom.Internal.SingleInstancePerformanceCounterCategoryDescriptor.CreateManager()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkPerfCounterManager..ctor()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess..cctor()
   — End of inner exception stack trace —
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.Initialize()
   at Microsoft.EnterpriseManagement.Mom.Sdk.Service.SdkSubService.AsyncStartCallback(Object notUsed)
   at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="OpsMgr SDK Service" />
    <EventID Qualifiers="49152">26380</EventID>
    <Level>2</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2011-01-07T19:25:48.000Z" />
    <EventRecordID>10025424</EventRecordID>
    <Channel>Operations Manager</Channel>
    <Computer>INFMGT52.accessgeneral.com</Computer>
    <Security />
  </System>
  <EventData>
    <Data>System.TypeInitializationException: The type initializer for ‘Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess’ threw an exception. —&gt; System.InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.
   at System.Diagnostics.PerformanceCounter.Initialize()
   at System.Diagnostics.PerformanceCounter.set_RawValue(Int64 value)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager.CreateCounter(String categoryName, String counterName)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager.CreateCounters(PerformanceCounterCategoryDescriptor categoryInfo)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager..ctor(String instanceName, PerformanceCounterCategoryDescriptor categoryInfo)
   at Microsoft.EnterpriseManagement.Mom.Internal.SingleInstancePerformanceCounterCategoryDescriptor.CreateManager()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkPerfCounterManager..ctor()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess..cctor()
   — End of inner exception stack trace —
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.Initialize()
   at Microsoft.EnterpriseManagement.Mom.Sdk.Service.SdkSubService.AsyncStartCallback(Object notUsed)
   at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)</Data>
  </EventData>
</Event>

After some research I found this article from Microsoft that contains instructions on how to fix this problem.

Dashboarding using the SCCM Dashboard solution accelerator example

I will turn this into a more useful blog post when I have a few minutes but for now it is just a collection of links about dashboarding using the System Center Configuration Manager Solution Accelerator. The short story is that this runs on top of Sharepoint 3.0, which is free, meaning you can run this for free. It’s not specific to SCCM so you can dashboard any sql data you want! Cool!

http://technet.microsoft.com/en-us/library/ff369719.aspx

http://systemscentre.blogspot.com/search/label/Dashboard

http://garyhay.blogspot.com/2010/07/sccm-dashboard-queries-1.html

http://blogs.msdn.com/b/shitanshu/archive/2010/04/08/part-1-how-microsoft-it-using-configuration-manager-dashboard.aspx

http://blogs.msdn.com/b/shitanshu/archive/2010/04/11/part-2-using-configuration-manager-dashboard-for-software-update-deployment-readiness-patch-tuesday-checklist.aspx

http://blogs.msdn.com/b/shitanshu/archive/2010/05/02/part-3-sql-queries-used-for-creating-custom-configuration-manager-dashboard-in-microsoft-it.aspx

http://blogs.msdn.com/b/shitanshu/archive/2010/10/09/part-4-sql-queries-for-creating-configuration-manager-client-health-and-problem-management-dashboard.aspx

http://64.4.11.252/en-us/edge/system-center-configuration-manager-2007-dashboard-part-2-configuration.aspx?query=1

SCOM Windows 2008 R2 Printer MP

Ran across this custom MP by Kevin Holman. Nice! Thank you!

cleaning up the default management pack

SCOM has what I feel is a major bug in that it will allow you to save items (monitors, rules, overrides, etc.) in the default MP. Doing this is bad for a lot of reasons, and not only does SCOM allow you to do this, but it is the default option as well. In my case it turned out that an occasional lack of attention allowed me to do this and then removing MP’s later becomes a huge pain in the rear. Anyway I found this good article on how to clean up the mess.

SCOM: powershell run space failed to start

I have been getting these messages since day 1 and tried various things that didn’t work to resolve.

Below I am pasting an example rule with full text so that if someone is searching they will find it. This is one specific alert, I was having an issue with all non microsoft powershell scripted discoveries. For me this was 99% from the XSNMP SCOM Management Pack. To be clear the MP was not the cause of the problem, only the one that tried to run PS (and not work) the most.

PM

Log Name:
Operations Manager

Source:
Health Service Modules

Event Number:
22400

Level:
1

Logging Computer:

User:
N/A

Description:

Failed to run the PowerShell script due to exception below, this workflow will be unloaded. System.NullReferenceException: Object reference not set to an instance of an object. at System.Environment.GetEnvironmentVariable(String variable, EnvironmentVariableTarget target) at System.Management.Automation.ModuleIntrinsics.GetExpandedEnvironmentVariable(String name, EnvironmentVariableTarget target) at System.Management.Automation.ModuleIntrinsics.SetModulePath() at System.Management.Automation.ExecutionContext.InitializeCommon(AutomationEngine engine, PSHost hostInterface) at System.Management.Automation.AutomationEngine..ctor(PSHost hostInterface, RunspaceConfiguration runspaceConfiguration, InitialSessionState iss) at System.Management.Automation.Runspaces.LocalRunspace.DoOpenHelper() at System.Management.Automation.Runspaces.RunspaceBase.CoreOpen(Boolean syncCall) at Microsoft.EnterpriseManagement.Modules.PowerShell.RunspaceController.RunScript(String scriptName, String scriptBody, Dictionary`2 parameters, PowerShellOutputType outputType, Int32 serializationDepth, IModuleDebug iModuleDebug) at Microsoft.EnterpriseManagement.Modules.PowerShell.PowerShellProbeActionModule.RunScript(RunspaceController runspaceController) Script Name: MemoryPctUtil.ps1 One or more workflows were affected by this. Workflow name: xSNMP.Cisco.Rule.CollectMemoryPoolUtil Instance name: I/O Instance ID: {X} Management group: X

Event Data:

< DataItem type =" System.XmlData " time =" 2010-12-03T19:15:30.1742570-05:00 " sourceHealthServiceId =" X" >

< EventData >

< Data > X </ Data >

< Data > xSNMP.Cisco.Rule.CollectMemoryPoolUtil </ Data >

< Data > I/O </ Data >

< Data > {X} </ Data >

< Data > MemoryPctUtil.ps1 </ Data >

< Data > 300 </ Data >

< Data > System.NullReferenceException: Object reference not set to an instance of an object. at System.Environment.GetEnvironmentVariable(String variable, EnvironmentVariableTarget target) at System.Management.Automation.ModuleIntrinsics.GetExpandedEnvironmentVariable(String name, EnvironmentVariableTarget target) at System.Management.Automation.ModuleIntrinsics.SetModulePath() at System.Management.Automation.ExecutionContext.InitializeCommon(AutomationEngine engine, PSHost hostInterface) at System.Management.Automation.AutomationEngine..ctor(PSHost hostInterface, RunspaceConfiguration runspaceConfiguration, InitialSessionState iss) at System.Management.Automation.Runspaces.LocalRunspace.DoOpenHelper() at System.Management.Automation.Runspaces.RunspaceBase.CoreOpen(Boolean syncCall) at Microsoft.EnterpriseManagement.Modules.PowerShell.RunspaceController.RunScript(String scriptName, String scriptBody, Dictionary`2 parameters, PowerShellOutputType outputType, Int32 serializationDepth, IModuleDebug iModuleDebug) at Microsoft.EnterpriseManagement.Modules.PowerShell.PowerShellProbeActionModule.RunScript(RunspaceController runspaceController) </ Data >

< Data />

</ EventData >

</ DataItem >

 

Eventually the scripts would time out like this:

Time window start
12/7/2010 11:57:38 AM

Time window end
12/7/2010 12:02:37 PM

Time First
12/7/2010 11:57:38 AM

Time Last
12/7/2010 11:57:41 AM

Count
44

Context

Date and Time:
12/7/2010 11:57:41 AM

Log Name:
Operations Manager

Source:
Health Service Modules

Event Number:
22411

Level:
1

Logging Computer:
X

User:
N/A

Description:

The PowerShell script will be dropped because the it has been waiting in the queue for more than 10 minutes. Script Name: DiscoverInterfaceName.ps1 One or more workflows were affected by this. Workflow name: xSNMP.Discovery.InterfaceName Instance name: GigabitEthernet2/21 Instance ID: {X} Management group: X

Event Data:

< DataItem type =" System.XmlData " time =" 2010-12-07T11:57:41.4054873-05:00 " sourceHealthServiceId =" X" >

< EventData >

< Data > X </ Data >

< Data > xSNMP.Discovery.InterfaceName </ Data >

< Data > GigabitEthernet2/21 </ Data >

< Data > {X} </ Data >

< Data > DiscoverInterfaceName.ps1 </ Data >

< Data > 300 </ Data >

< Data > 10 </ Data >

< Data />

</ EventData >

</ DataItem >

 

The issue turned out to be  a permissions issue with the health service since it was trying to run these powershell scripts as local system. This article is the one that finally jogged my memory appropriately. Thank you to the author!