Cisco UC Database Replication Issues

Try the following commands

utilts network connectivity
show network cluster
utils dbreplication reset all

If the databases are out of sync tru this

utils dbreplication stop all (from Pub)
utils dbreplication forcedatasyncsub <subnodename> (from Pub)
utils system restart (restart of the Subscribers to which sync is done after around 10 mins)

If the database replication is not setup and the reset commands did not

utils dbreplication rebuild all

CUCM Compatibility Tool
http://www.cisco.com/c/en/us/td/docs/voice_ip_comm/uc_system/unified/communications/system/Compatibility/CSR-Compatibility-Matrix.html

Pre-Version 10.0 http://tools.cisco.com/ITDIT/vtgsca/VTGServlet

Cisco Demo Licenses

CUCM Service Activation

CDRs

Extension Mobilty - Remote Logout / Login
Extension Mobility - bulk assign Owner ID to logged in user

http://www.cisco.com/c/en/us/support/docs/voice-unified-communications/bulk-administration-tool/110967-bat-00.html

  • How to do this in Bulk!
  • Get all Device Profiles Logged into Phones
  • Export (copy and paste) the details from Actively logged in Devices and add to worksheet in EXCEL with Just two Columns, MAC & OwnerUserID - (worksheet name “OwnerUserID”)
  • Do an Export All Details of the specific phones (e.g. 7841) and export add to another worksheet in the same excel
  • In the All Details export Remove all phones which already have an associated User
  • With Phones which don't have a userID , add the following Vlookup command “=VLOOKUP,A2,OwnerUserID!A:B,2”
  • This will auto populate the Owner User ID Fields with the correct user ID which is logged into it.
  • Export the file as CSV.
  • Import the file into CUCM
  • Run BAT Insert Phones - All Details,
  • select Override the existing config
  • Also Check all the “Delete all existing…” (as the config wil be readded and you will have phones with two services etc. if you don't!
  • Run !

An unsupported way to do this is documented below. Only do this in Test lab!!! Reference: https://crystalclearinsanity.wordpress.com/2014/03/07/cucm-set-device-owner-id-to-em-logged-in-user-id-via-sql/

run sql update device set fkenduser=(select fkenduser from extensionmobilitydynamic where fkdevice=device.pkid) where name like 'SEP%' and fkenduser IS NULL
run sql update device set fkenduser=(select fkenduser from extensionmobilitydynamic where fkdevice=device.pkid) where name like 'SEP%'

Bulk EM Login-Logout Script

@echo off
: Gerard O'Rourke
: Date 05/09/2016

SET WGETFOLDER=CC:\X-Support\Utilities\ccm-bulk-em-login\wget
SET CALLMANAGER=uc-cucm-pub
SET INPUTFILE=Login-Config.txt
SET LOGINLOGFILE=BulkLogin.log
SET LOGOUTLOGFILE=BulkLogout.log

goto start

echo.
echo Login / Logout Program for Cisco CallManager 
echo.
echo Note: This batch files requires WGET for Windows.
echo WGET for Windows is available at http://gnuwin32.sourceforge.net/packages/wget.htm
echo.
echo The Format of Input file should be a comma separated file 
echo with the MAC, username and PIN as the following example shows:
echo.
echo ======= Example Input File ========
echo SEP001122AABB01,myusername1,5551234 
echo SEP001122AABB02,myusername2,5551234 
echo ===================================
echo.
echo Note: if only using the  program to log out phones, no need for column 2 or 3 in the config file
echo. 



echo ==================================================

SET /P WGETFOLDER=Please Set WGET Folder if not already in Windows PATH [%WGETFOLDER%]?
SET /P CALLMANAGER=Please Set IP Address of the Cisco CallManager [%CALLMANAGER%]?
SET /P INPUTFILE=Please Set the name of the Input File [%INPUTFILE%]?


:start

SET OPTION=3
SET CONFIRM=N

cls
echo ==================================================

echo.
echo 1. Login Phones
echo 2. Logout Phones
echo 3. Exit
echo.

SET /P OPTION=Select Menu Option [%option%]?

If "%OPTION%" =="1" goto LoginPhones
If "%OPTION%" =="2" goto LogoutPhones 

echo "Exiting Program - Nothing Implemented."
pause
exit


:LoginPhones
cls
echo.
echo About to start Login Phones
SET /P CONFIRM=Are you sure you want to continue (Y/N) [%CONFIRM%]?

If "%CONFIRM%" =="y" goto LoginPhonesStart
If "%CONFIRM%" =="Y" goto LoginPhonesStart

exit

:LoginPhonesStart

FOR /F "tokens=1,2,3 delims=," %%a IN (%INPUTFILE%) DO (

echo wget -a %loginlogfile% --delete-after http://%CallManager%:8080/emapp/EMAppServlet?device=%%a^&userid=%%b^&seq=%%c
wget -a %loginlogfile% --delete-after http://%CallManager%:8080/emapp/EMAppServlet?device=%%a^&userid=%%b^&seq=%%c

ping 127.0.0.1 -n 2 >NUL | REM This pauses for a second on Win7 or 2 on XP
)
echo.
echo.
echo.
echo ****** FINSHED Phones Logged in *******
pause
exit

:LogoutPhones
cls
echo.
echo About to start Logout Phones
SET /P CONFIRM=Are you sure you want to continue (Y/N) [%CONFIRM%]?

If "%CONFIRM%" =="y" goto LogoutPhonesStart
If "%CONFIRM%" =="Y" goto LogoutPhonesStart

exit

:LogoutPhonesStart

FOR /F "delims=," %%a IN (%INPUTFILE%) DO (
echo wget -a %logoutlogfile% --delete-after http://%CallManager%:8080/emapp/EMAppServlet?device=%%a^&doLogout=true 
wget -a %logoutlogfile% --delete-after http://%CallManager%:8080/emapp/EMAppServlet?device=%%a^&doLogout=true 
ping 127.0.0.1 -n 2 >NUL | REM This pauses for a second on Win7 or 2 on XP
)

echo.
echo.
echo.
echo ****** FINSHED Phones Logged out *******
pause
exit

To Log them out:

http://<serveraddress>:8080/emapp/EMAppServlet?device=SEP001122AABBCC&doLogout=true 

To log them back in again:

http://<serveraddress>:8080/emapp/EMAppServlet?device=SEP001122AABBCC&userid=myusername&seq=5551234 

CUCM CDR's
origReleaseCause - if set to 16 - CUCM user hung up call
destReleaseCause - If set to 16 - Far end / Customer hung up the call.

3rd Party SIP Phones
Secondary Dial tone issues & SQL read on CUCM

LDAP Filters in CUCM

https://supportforums.cisco.com/document/112451/ldap-custom-filter-cucm-8x
Filter to import only users who have a telephony number assigned.

(&(objectclass=user)(telephonenumber=*))

Users who are a member of a group

(&(objectClass=user)(memberof=CN=SampleGroup1,OU=CustomGroups,DC=ucce-dev,DC=mydomain,DC=pri))

Users who are NOT a member of a group

(&(objectClass=user)(!(memberof=CN=SampleGroup1,OU=CustomGroups,DC=ucce-dev,DC=mydomain,DC=pri)))

This Example filters out non active accounts. for more info see this AD User Properties

(&(objectclass=user)(!(objectclass=Computer))(!(UserAccountControl:1.2.840.113556.1.4.803:=2))(|(ipPhone=*)(telephonenumber=*)(mobile=*)(otherTelephone=*))) LDAPFIL-ESB-AD-All (&(objectclass=user)(!(objectclass=Computer))(!(UserAccountControl:1.2.840.113556.1.4.803:=2)))

SSO
Single Sign On

Using IOS Router for ntp & dns for install
DNS server for CUCM lab install

Testing your CUCM Security Password Testing your CUCM Security Password

Multi Server Certificate
http://www.cisco.com/c/en/us/support/docs/unified-communications/unified-communications-manager-callmanager/118731-configure-san-00.html

Auto Registration Phone Config

Music On Hold Download

Phone Reset on Call Answer

https://supportforums.cisco.com/document/115991/reason-codes-lastoutofserviceinformation

I had an issue where a Phone Resets on Call Answer. Reviewing the Phone Logs you could see it got a “LastOutOfService” Code of '12'. So Call Manager was sending a Reset. But when you reviewed the packet capture arriving at the phone and a separate capture leaving the CUCM, it was clear they were not the same. i.e. CUCM was NOT resetting the phone.

The Phone Reset when it Receieved a packet from CUCM of a TCP FIN-ACK. But CUCM never set this. It sent a 200 OK with SDP as expected on call answer.

But a firewall between CUCM and the Phone received the SIP 200 OK and due to IPS, sent a FIN ACK back to CUCM and to the Phone (and never sent the 200 OK to the phone). i.e. So it 'appears' that the TCP FIN-ACK sent by the phone (to CUCM) and appears that CUCM sent the Phone the FIN ACK.

Configuring the (Checkpoint in this case) with SIP domain for the necessary devices, corrected the issue.

  • utils os secure permisssive
  • Right Click on Virtual Machine on vSphere and select install VMware tool
  • utils vmtools resfresh
  • wait for installation to complete and VMware tools to be Green
  • utils os secure enforce

Retrieving Packet Capture using CLI

Troubleshoot CUCM Web (GUI) Issues

Useful CLI Troubleshooting commands

utils diagnose test
utils ntp status
show process load cpu
show process load memory
show process using-most cpu
show process using-most memory
utils core active list

CUCM Disk Usuage & Trace Log Alerts

To Review Disk Usuage

 show diskusage activelog directory sort

Use the below command to delete logs files.

file delete activelog /cm/trace/ccm/sdi/* noconfirm

More Info
http://docwiki.cisco.com/wiki/Troubleshooting_DiskSpace_and_Partition_related_alerts

[RTMT-ALERT-StandAloneCluster] LogPartitionLowWaterMarkExceeded

The defaults are 95 percent for high watermark and 90 percent for low watermark.)
http://docwiki.cisco.com/wiki/Unified_CM_L2_Upgrade_Disk_Space_issues

http://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cucm/service/10_0_1/rtmt/CUCM_BK_CA30A928_00_cisco-unified-rtmt-administration-100/CUCM_BK_CA30A928_00_cisco-unified-rtmt-administration-100_chapter_010.html

If the percentage of disk usage is above the high water mark that you configured, the system sends an alarm message to syslog, generates a corresponding alert in RTMT Alert Central, and automatically purges log files until the value reaches the low water mark.

Digit Digit Analysis Debugging on CUCM

file tail activelog cm/trace/ccm/sdl/<filename> regexp "nalysis"
admin:file tail activelog cm/trace/ccm/sdl/SDL001_100_000611.txt.gzo regexp "nalysis"
00111483.000 |15:26:05.315 |SdlSig   |DaReq                                  |wait                           |Da(1,100,211,1)                  |Cdcc(1,100,219,193)              |1,100,10,1.1520^192.168.1.30^*           |[R:N-H:0,N:0,L:0,V:0,Z:0,D:0]  CI=19453978 Fqdn=ti=1nd=12000pi=0si1 Cgpn=tn=0npi=0ti=1nd=12000pi=1si0 DialedNum=tn=0npi=1ti=1nd=13003User=13003Host=192.168.1.170Port=5060PassWord=Madder=Transport=4mDisplayName=RawUrl=sip:13003@192.168.1.170OrigPort=0pi=0si1 requestID=0 DigitAnalysisComplexity=0 CallingUser= IgnoreIntercept=0
00111483.001 |15:26:05.315 |AppInfo  |Digit Analysis: star_DaReq: daReq.partitionSearchSpace(:cc218c90-a780-503c-062e-90ba0bdd47f1), filteredPartitionSearchSpaceString(CallRecord_External_P:Internal_P), partitionSearchSpaceString(CallRecord_External_P:Internal_P)
00111483.002 |15:26:05.315 |AppInfo  |Digit Analysis: Host Address=192.168.1.170 MATCHES this node's IPv4 address.
00111483.003 |15:26:05.315 |AppInfo  |Digit Analysis: star_DaReq: Matching SIP URL, Numeric User, user=13003
00111483.004 |15:26:05.315 |AppInfo  |Digit Analysis: getDaRes data: daRes.ssType=[0] Intercept DAMR.sstype=[0], TPcount=[0], DAMR.NotifyCount=[0], DaRes.NotifyCount=[0]
00111483.005 |15:26:05.315 |AppInfo  |Digit Analysis: getDaRes - Remote Destination [] isURI[1]
00111483.006 |15:26:05.315 |AppInfo  |Digit analysis: patternUsage=2
00111483.007 |15:26:05.315 |AppInfo  |Digit analysis: match(pi="2", fqcn="12000", cn="12000",plv="5", pss="CallRecord_External_P:Internal_P", TodFilteredPss="CallRecord_External_P:Internal_P", dd="13003",dac="0")
00111483.008 |15:26:05.315 |AppInfo  |Digit analysis: analysis results
00111485.001 |15:26:05.315 |AppInfo  |Digit analysis: wait_DmPidRes- Partition=[] Pattern=[13003] Where=[],cmDeviceType=[UserDevice], OutsideDialtone =[0], DeviceOverride=[0], PID=LineControl(1,100,174,46),CI=[19453978],Sender=Cdcc(1,100,219,193)
00111539.000 |15:26:05.396 |SdlSig   |DaReq                                  |wait                           |Da(1,100,211,1)                  |Cdcc(1,100,219,193)              |1,200,13,2.856^192.168.1.178^13003       |[R:N-H:0,N:5,L:0,V:0,Z:0,D:0]  CI=19453978 Fqdn=ti=1nd=12000pi=0si1 Cgpn=tn=0npi=0ti=1nd=12000pi=0si1 DialedNum=ti=1nd=1309901pi=0si1 requestID=0 DigitAnalysisComplexity=0 CallingUser= IgnoreIntercept=0
00111539.001 |15:26:05.396 |AppInfo  |Digit Analysis: star_DaReq: daReq.partitionSearchSpace(:8d071003-57cb-2996-2287-49c4454fc689), filteredPartitionSearchSpaceString(Internal_P:Restricted_P), partitionSearchSpaceString(Internal_P:Restricted_P)
00111539.002 |15:26:05.396 |AppInfo  |Digit Analysis: star_DaReq: Matching Legacy Numeric, digits=1309901
00111539.003 |15:26:05.396 |AppInfo  |Digit Analysis: getDaRes data: daRes.ssType=[0] Intercept DAMR.sstype=[0], TPcount=[0], DAMR.NotifyCount=[0], DaRes.NotifyCount=[0]
00111539.004 |15:26:05.396 |AppInfo  |Digit Analysis: getDaRes - Remote Destination [] isURI[1]
00111539.005 |15:26:05.396 |AppInfo  |Digit analysis: patternUsage=2
00111539.007 |15:26:05.396 |AppInfo  |Digit analysis: analysis results
00111550.001 |15:26:05.407 |AppInfo  |Digit analysis: wait_DmPidRes- Partition=[e1890ede-8077-4a9b-9d91-d0498cbd49b9] Pattern=[1309901] Where=[],cmDeviceType=[UserDevice], OutsideDialtone =[0], DeviceOverride=[0], PID=LineControl(1,100,174,53),CI=[19453978],Sender=Cdcc(1,100,219,193)
00112164.000 |15:26:09.769 |SdlSig   |DaReq                                  |wait                           |Da(1,100,211,1)                  |Cdcc(1,100,219,195)              |1,100,10,1.1525^192.168.1.30^*           |[R:N-H:0,N:1,L:0,V:0,Z:0,D:0]  CI=19453982 Fqdn=ti=1nd=1309901pi=0si1 Cgpn=tn=0npi=0ti=1nd=1309901pi=1si0 DialedNum=tn=0npi=1ti=1nd=12029pi=0si1 requestID=0 DigitAnalysisComplexity=0 CallingUser= IgnoreIntercept=0
00112164.001 |15:26:09.769 |AppInfo  |Digit Analysis: star_DaReq: daReq.partitionSearchSpace(:72d47258-31ca-9429-3068-363fdd7eb2bc), filteredPartitionSearchSpaceString(Internal_P), partitionSearchSpaceString(Internal_P)
00112164.002 |15:26:09.769 |AppInfo  |Digit Analysis: star_DaReq: Matching Legacy Numeric, digits=12029
00112164.003 |15:26:09.769 |AppInfo  |Digit Analysis: getDaRes data: daRes.ssType=[0] Intercept DAMR.sstype=[0], TPcount=[0], DAMR.NotifyCount=[0], DaRes.NotifyCount=[0]
00112164.004 |15:26:09.769 |AppInfo  |Digit Analysis: getDaRes - Remote Destination [] isURI[1]
00112164.005 |15:26:09.769 |AppInfo  |Digit analysis: patternUsage=2
00112164.006 |15:26:09.769 |AppInfo  |Digit analysis: match(pi="2",fqcn="1309901", cn="1309901", plv="5", pss="Internal_P", TodFilteredPss="Internal_P", dd="12029",dac="0")
00112164.007 |15:26:09.769 |AppInfo  |Digit analysis: potentialMatches=NoPotentialMatchesExist

Note: Open SSL can be downloaded here (note install the light version)

openssl s_client -connect myldapsserver.domain.com:636

Part of the output of this file will be the Base-64 encoded .cer file that was presented for LDAPS.
Copy and paste into notepad beginning at “–Begin Certificate–” through “—End Certificate—”
save as a .cer

Double-click on the certificate file and you will now be viewing the certificate presented for LDAPS.

Reference: https://social.technet.microsoft.com/Forums/windowsserver/en-US/06c2a9cc-bd43-4bcd-9ade-b121c797d1f9/how-to-check-what-certificate-is-being-used-for-ssl-ldaps-connections?forum=winserversecurity

Is your AD LDAP enabled for SSL?
Reference: https://support.microsoft.com/en-us/help/321051/how-to-enable-ldap-over-ssl-with-a-third-party-certification-authority

IP address of CUCM phones

Login to each CUCM server which phones register to and run the following command:

show risdb query phone

To view Model

run sql select enum, name from type model

Secondary / Outside Dialtone 'Missing'

Find the dial pattern where outside dialtone is NOT set to “t” and the route pattern starts with '0'

run sql SELECT dnorpattern, tkpatternusage, routepartition.name AS partition FROM numplan JOIN routepartition on routepartition.pkid = numplan.fkroutepartition WHERE outsidedialtone = "f" AND dnorpattern like "0%" AND tkpatternusage <> "15"

https://www.youtube.com/watch?v=785jmTgiz1U

  • Extract isolinux\isolinux.bin
  • Generate BootInfoTable
  • Load Boot file (isolinux.bin)