Solved

Many functions down in our PROD environment any help appreciated

  • 3 November 2023
  • 5 replies
  • 53 views

Badge

1. Receipts not processing with functions unresponsive - can't release.
2. Can't complete shipments
3. All 'export to excel' functions are not working
4. Can't create POs
5. Can't download postings or run reports.

 

This all occurred overnight with no changes to our env.

 

Only occurring in our PROD env. the DEV. env. is working fine.

 

The error in the trace is the same for all:
at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception error)

 

icon

Best answer by Mike Gifford 3 November 2023, 17:09

View original

5 replies

Badge

Trace when trying to export to excel a User.


SELECT [r_MUIUserScreenRelevance].[UserID] AS [MUIUserScreenRelevance_UserID], [r_MUIUserScreenRelevance].[NodeID] AS [MUIUserScreenRelevance_NodeID], [r_MUIUserScreenRelevance].[Relevance] AS [MUIUserScreenRelevance_Relevance]
FROM [MUIUserScreenRelevance] [r_MUIUserScreenRelevance]
WHERE [r_MUIUserScreenRelevance].[UserID] = CONVERT(UNIQUEIDENTIFIER, '90ba6009-e2ed-4c7a-be62-a48abe825838') AND ( [r_MUIUserScreenRelevance].[CompanyID] = 4)
ORDER BY [r_MUIUserScreenRelevance].[UserID], [r_MUIUserScreenRelevance].[NodeID] OPTION(OPTIMIZE FOR UNKNOWN)

   at Serilog.Core.Logger.Write(LogEventLevel level, Exception exception, String messageTemplate, Object[] propertyValues)
   at Serilog.Core.Logger.Information[T](String messageTemplate, T propertyValue)
   at PX.Data.PXDatabaseProviderBase.Select(PXGraph graph, BqlCommand command, Int64 skip, Int64 topCount, PXView view, PXDataValue[] pars)
   at PX.Data.PXDatabaseProvider.Select(PXGraph graph, BqlCommand command, Int64 topCount, PXView view, PXDataValue[] pars)
   at PX.Data.PXGraph.ProviderSelect(BqlCommand command, Int32 topCount, PXView view, PXDataValue[] pars)
   at PX.Data.PXView.GetResult(Object[] parameters, PXFilterRow[] filters, Boolean reverseOrder, Int32 topCount, PXSearchColumn[] sorts, Boolean& overrideSort, Boolean& extFilter)
   at PX.Data.SQLTree.SQLinqView.GetResult(Object[] parameters, PXFilterRow[] filters, Boolean reverseOrder, Int32 topCount, PXSearchColumn[] sorts, Boolean& overrideSort, Boolean& extFilter)
   at PX.Data.PXView.Select(Object[] currents, Object[] parameters, Object[] searches, String[] sortcolumns, Boolean[] descendings, PXFilterRow[] filters, Int32& startRow, Int32 maximumRows, Int32& totalRows)
   at PX.Data.SQLTree.SQLinqExecutor.ExecuteCollection[T](Query query, SQLinqBqlCommandInfo info, PXGraph graph, SQLinqExecutor linqExecutor)
   at PX.Data.SQLTree.SQLinqExecutor.ExecuteCollection[T](Query query, PXGraph graph)
   at PX.Data.SQLTree.SQLinqExecutor.ExecuteCollection[T](QueryModel queryModel)
   at Remotion.Linq.Clauses.StreamedData.StreamedSequenceInfo.ExecuteQueryModel(QueryModel queryModel, IQueryExecutor executor)
   at Remotion.Linq.QueryProviderBase.System.Linq.IQueryProvider.Execute[TResult](Expression expression)
   at Remotion.Linq.QueryableBase`1.GetEnumerator()
   at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at PX.Web.UI.Frameset.Services.SiteMapPersonalizationService.GetSiteMapNodeRelevance()
   at lambda_method(Closure , Object , Object[] )
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass6_2.b__2(Object instance, Object[] methodParameters)
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)
   at System.Web.Http.Controllers.ApiControllerActionInvoker.d__1.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Web.Http.Controllers.ApiControllerActionInvoker.InvokeActionAsyncCore(HttpActionContext actionContext, CancellationToken cancellationToken)
   at System.Web.Http.Controllers.ApiControllerActionInvoker.InvokeActionAsync(HttpActionContext actionContext, CancellationToken cancellationToken)
   at System.Web.Http.Controllers.ActionFilterResult.<>c.b__5_0(ActionInvoker innerInvoker)
   at System.Web.Http.Controllers.ActionFilterResult.<>c__DisplayClass7_0`1.b__0()
   at System.Web.Http.Filters.ActionFilterAttribute.d__6.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Web.Http.Filters.ActionFilterAttribute.CallOnActionExecutedAsync(HttpActionContext actionContext, CancellationToken cancellationToken, Func`1 continuation)
   at System.Web.Http.Filters.ActionFilterAttribute.d__5.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Web.Http.Filters.ActionFilterAttribute.ExecuteActionFilterAsyncCore(HttpActionContext actionContext, CancellationToken cancellationToken, Func`1 continuation)
   at System.Web.Http.Filters.ActionFilterAttribute.System.Web.Http.Filters.IActionFilter.ExecuteActionFilterAsync(HttpActionContext actionContext, CancellationToken cancellationToken, Func`1 continuation)
   at System.Web.Http.Controllers.ActionFilterResult.d__5.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Web.Http.Controllers.ActionFilterResult.ExecuteAsync(CancellationToken cancellationToken)
   at System.Web.Http.Filters.AuthorizationFilterAttribute.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Web.Http.Filters.AuthorizationFilterAttribute.ExecuteAuthorizationFilterAsyncCore(HttpActionContext actionContext, CancellationToken cancellationToken, Func`1 continuation)
   at System.Web.Http.Filters.AuthorizationFilterAttribute.System.Web.Http.Filters.IAuthorizationFilter.ExecuteAuthorizationFilterAsync(HttpActionContext actionContext, CancellationToken cancellationToken, Func`1 continuation)
   at System.Web.Http.Controllers.AuthorizationFilterResult.ExecuteAsync(CancellationToken cancellationToken)
   at System.Web.Http.ApiController.ExecuteAsync(HttpControllerContext controllerContext, CancellationToken cancellationToken)
   at System.Web.Http.Dispatcher.HttpControllerDispatcher.d__15.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Web.Http.Dispatcher.HttpControllerDispatcher.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpMessageInvoker.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Web.Http.Dispatcher.HttpRoutingDispatcher.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at PX.Api.Compression.ServerCompressionHandler.d__10.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
   at PX.Api.Compression.ServerCompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Web.Http.HttpServer.d__24.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Web.Http.HttpServer.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpMessageInvoker.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Web.Http.Owin.HttpMessageHandlerAdapter.d__20.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Web.Http.Owin.HttpMessageHandlerAdapter.InvokeCore(IOwinContext context, IOwinRequest owinRequest, IOwinResponse owinResponse)
   at PX.Owin.Startup.<b__12_5>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at PX.Owin.Startup.b__12_5(IOwinContext ctx, Func`1 next)
   at PX.Logging.Enrichers.OwinExtensions.<>c.<b__0_0>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at PX.Logging.Enrichers.OwinExtensions.<>c.b__0_0(IOwinContext ctx, Func`1 next)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.d__3.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Mapping.MapWhenMiddleware.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.d__7.MoveNext()
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.RunApp(Func`2 entryPoint, IDictionary`2 environment, TaskCompletionSource`1 tcs, StageAsyncResult result)
   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.BeginEvent(Object sender, EventArgs e, AsyncCallback cb, Object extradata)
   at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.<>c__DisplayClass285_0.b__0()
   at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
   at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception error)
   at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext context, AsyncCallback cb)
   at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context)
   at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)
   at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)

Userlevel 4
Badge

My gut says to reset/restart IIS and check on condition of SQL Server as a starting point, but that would also depend on your environment (ie SaaS, self hosted or cloud hosted) and whether you have any Processing Nodes or other websites within IIS.

 

I know this is the equivalent of “have you rebooted windows” - but it works, so..

Badge

“Restart Application” in all sites resolved the issues.

Still looking for root cause.

Userlevel 7
Badge +12

“Restart Application” in all sites resolved the issues.

Still looking for root cause.

Restart application will help this type of issue,

I think we cannot get a root cause for these typeof  issue,

 

Window IIS restart will help to clear caches or republish the cusotmization packages clear this issues

Userlevel 3
Badge

Try to restart the System and if still not worked then clear cache and try to open in another browser.

I hope it would work

Reply


About Acumatica ERP system
Acumatica Cloud ERP provides the best business management solution for transforming your company to thrive in the new digital economy. Built on a future-proof platform with open architecture for rapid integrations, scalability, and ease of use, Acumatica delivers unparalleled value to small and midmarket organizations. Connected Business. Delivered.
© 2008 — 2024  Acumatica, Inc. All rights reserved